Gatling

Always on the lookout for new tools that can help in the testing of APIs, one that I recently found is called Gatling. It is an API performance testing framework written in Scala. We are not currently using it, though I think it would be very beneficial to have in the testing pipeline and is simple to set up.

Comparison to JMeter

JMeter seems to be the go-to for performance testing as it has been around for a while. Being a developer though, I like to code, and that seems to be the big difference between the two frameworks. Gatling is a library that has its own API that you can code to. JMeter has a UI that you use to generate your scripts. Both frameworks have “recording” functionality that you can capture your http events and replay them. However, I think the benefit of code is re-usability and done correctly will allow you to create your scripts much faster, and be more fun in the process.

Sample Code

Below is a short example of a test that authenticates and hits an API for a set of licenses.

class HttpSimulation1 extends Simulation {


	object Actions {
    def storeAuthData(session: Session, datakey: String): String = {
      val data = session.get(datakey).as[String];      
      val jValue = scala.util.parsing.json.JSON.parseFull(data).get;

      val map : Map[Any,Any] = jValue.asInstanceOf[Map[Any,Any]];
      val dataMap = map.get("data").get.asInstanceOf[Map[Any,Any]];
      val attributeMap = dataMap.get("attributes").get.asInstanceOf[Map[Any,Any]];
      val token : String = attributeMap.get("token").get.asInstanceOf[String];

      return token;
    }
  }


  val feeder = csv("users.csv").circular;

  object Auth {

    var pickUser = feed(feeder);


    var login = exec{ session =>
    val u = session("username").as[String];
    val p = session("password").as[String]
    val str = u + ":" + p
    session.set("jwtToken",
	Base64.getEncoder.encodeToString(str.getBytes(StandardCharsets.UTF_8)));

    }.exec(

        http("jwt")          
            .post("http://localhost:8083/api/jwt")
          .headers(Map(
            "Content-Type" -> "application/vnd.api+json; charset=UTF-8"
          ))
          .body(StringBody(
            JsonConverter.toJson(Map(
              "data" -> Map(
                "attributes" -> Map(
                  "token" -> "${jwtToken}"
                ),
                "type" -> "user"

              )
            ))

          ))

          .check(status.is(200))
          .check(bodyString.saveAs("authdata"))
        )
      .exec{
        session =>
          val token = Actions.storeAuthData(session, "authdata")
          session.set("auth_token",token);
      }


  }

  object CompanyApi {
    val getCompanyProfile = exec(
      http("companyProfile")
        .get("https://localhost/api/companyProfile/${alias}")
        .headers(Map(
          "Content-Type" -> "application/vnd.api+json; charset=UTF-8"
        ))
        .check(status.is(200)))
      .pause(5,15)

    val getCompanyLicenses = exec(
      http("companyLicenses")
        .get("http://localhost/api/companyLicenses/${companyId}")
        .headers(Map(
          "Content-Type" -> "application/vnd.api+json; charset=UTF-8",
          "Authorization" -> "${auth_token}"
        ))
        .check(status.is(200)))
      .pause(0,5)

  }


  val theHttpProtocolBuilder = http
    .baseURL("https://computer-database.gatling.io")

  val theScenarioBuilder = scenario("Scenario1")
    .exec(Auth.pickUser,
      Auth.login,
      CompanyApi.getCompanyLicenses
    )

  setUp(
    theScenarioBuilder.inject(rampUsers(100) over (5 seconds))
  ).protocols(theHttpProtocolBuilder)
   .assertions(global.failedRequests.percent.is(0))
}


}

In the code above, we have two Objects, Auth and CompanyApi. Auth defines methods for authentication and CompanyApi defines methods around company methods. After we authenticate we store the token in a session map that can be retrieved in additional methods. We also define different “Scenarios” that we want to test. In this case Scenario1 authenticates and makes a call to get a company’s licenses. We store our users’ usernames/passwords in a csv file that a reader will continuously loop over to pass in new users for testing. In the setup section, we tell Gatling to ramp up to 100 users over 5 seconds and that we expect to have 0 failed requests.

Building

Gatling has a plugin for Maven that can generate reports and fail/pass based on the assertions specified.

mvn gatling:test

will have the following output

======================================================================
---- Global Information ----------------------------------------------
> request count                              200 (OK=197    KO=3     )
> min response time                           27 (OK=27     KO=119   )
> max response time                          463 (OK=434    KO=463   )
> mean response time                         143 (OK=141    KO=299   )
> std deviation                              104 (OK=101    KO=119   )
> response time 50th percentile              119 (OK=116    KO=250   )
> response time 75th percentile              212 (OK=211    KO=357   )
> response time 95th percentile              368 (OK=366    KO=442   )
> response time 99th percentile              427 (OK=400    KO=459   )
> mean requests/sec                          9.5 (OK=9.38   KO=0.143 )
---- Response Time Distribution --------------------------------------
> t < 800 ms                                           197 (  99%)
> 800 ms < t < 1200 ms                                   0 (  0%)
> t > 1200 ms                                            0 (  0%)
> failed                                                 3 (  2%)
---- Errors -----------------------------------------------------------
status.find.is(200), but actually found 500 3 (100.0%)
=======================================================================

Learn more about Gatling here. https://gatling.io/