☰
Navigation :

Scala (DSL)

The recommended way to interact with WARP is through the Scala DSL. This API provides a richer feature set than the Java API, including the ability to register custom MeasurementCollector and Arbiter instances, and add new tags in the form of String metadata that will be persisted.

The DSL is implemented by using an immutable case class, ExecutionConfig to hold all configuration parameters, such as number of invocations, warmups, threadpool size, etc. We give sane defaults to all parameters. Calls to the various DSL methods invoke the compiler-generated copy method to build up new instances of ExecutionConfig: 

1
2

import com.workday.warp.dsl._
val config: ExecutionConfig = using invocations 32 threads 4 distribution GaussianDistribution(50, 10)

Note, however, that the DSL itself manages the measurement lifecycle. Thus, we do not recommend using @WarpTest annotation together with the DSL, as that would lead to doubly measured tests. The DSL can be especially useful in cases where users already make heavy use of BeforeEach/AfterEach hooks. @WarpTest annotation is implemented using JUnit before/after hooks, the order of which cannot be controlled. Thus, it is possible that tests using @WarpTest will have extra overhead from other hooks included in their measurement metadata. The DSL is decoupled from JUnit and can be used with other JVM testing frameworks.

Finally, a call-by name block is passed to ExecutionConfig.measuring:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

import com.workday.warp.dsl._
import com.workday.warp.TestIdImplicits._
import org.junit.jupiter.api.{Test, TestInfo}

class ExampleSpec extends WarpJUnitSpec  {

  @Test
  def dslExample(testInfo: TestInfo): Unit = {
    using testId testInfo invocations 32 threads 4 measuring {
      val i: Int = 1
      Logger.info(s"result is ${i + i}")
    } should not exceed (2 seconds)
  }
}

Custom Arbiter and MeasurementCollector instances can be registered by calling the arbiters and collectors methods: 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

import com.workday.warp.dsl._
import com.workday.warp.TestIdImplicits._
import org.junit.jupiter.api.{Test, TestInfo}

class ExampleSpec extends WarpJUnitSpec {

  @Test
  def dslCollectors(testInfo: TestInfo): Unit = {
    // disables the existing default collectors, and registers a new collector
    using testId testInfo only these collectors {
      new SomeMeasurementCollector
    // registers two new arbiters
    } arbiters {
      List(new SomeArbiter, new SomeOtherArbiter)
    } measuring {
      someExperiment()
    } should not exceed (5 seconds)
  }
}

The arbiters and collectors methods both accept a call-by-name function that returns an Iterable. We use an implicit to lift a single instance into an Iterable type.

The threshold defined by the should not exceed syntax is implemented as a scalatest Matcher[Duration].

DSL Operations

The DSL provides a flexible way to describe experimental setups for conducting repeated trials, and supports the following operations:

DSL Return Type

After we have constructed an ExecutionConfig that fits the needs of our experiment, we call the measuring (or measure, a synonym) method with the call-by-name function f that we want to measure. The return type of measuring is List[TrialResult[T]], where T is the return type of f. This gives us access to the returned values of each invocation of f if they are needed for further analysis.

Additionally, TrialResult holds a TestExecutionRow corresponding to the newly written row in our database, and the measured response time of the trial.

For example, suppose we are interested in measuring the performance of List.fill[Int] construction:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

import com.workday.warp.dsl._
import com.workday.warp.utils.Implicits._
import com.workday.warp.TestIdImplicits._
import org.junit.jupiter.api.{Test, TestInfo}

class ExampleSpec extends WarpJUnitSpec {

  @Test
  def listFill(testInfo: TestInfo): Unit = {
    // measure creating a list of 1000 0's
    val results: Seq[TrialResult[List[Int]]] = using testId testInfo invocations 8 measure { List.fill(1000)(0) }

    // make some assertions about the created lists
    results should have length 8
    results.head.maybeResult should not be empty

    for {
      result <- results
      list <- result.maybeResult
      element <- list
    } element should be (0)
  }
}

After measurement has been completed, we are able to access the return values of the function being measured.

The DSL provides a flexible way to customize the execution schedule of your experiment, including adding new measurement collectors and arbiters for defining failure criteria.