Welcome to part two of our series. In part one, we covered testing interaction between different EV ecosystem components as well as simulating EV and charge station events. As previously mentioned, ensuring a standardized charging ecosystem is key as the adoption of EVs rises. Here in part two, we will delve deeper into testing OCPP implementation and its compliances. Understanding this is crucial to ensure that our software aligns with the standard.
OCPP 2.0.1 implementation with an EV client
As previously described in part one of the blog, OCPP is a communication protocol that’s used to exchange messages between multiple charge stations and a CSMS. This protocol was implemented as part of an engagement with an EV client.
Before Thoughtworks partnered with the EV client, the client had a need for compliance with the OCPP protocol and how the protocol was interpreted by 3rd party services. They then found our open-source CSMS (MaEVe) and liked our engineering practices and the fact that we included an implementation of OCPP 2.0.1 for MaEVe. They were introduced to Thoughtworks through an existing connection. Their aims were primarily driven by compliance with regulations. In the US there was some funding opportunity with the Nationalmailch Electric Vehicle Infrastructure (NEVI). So the idea was that complying with OCPP 2.0.1 would help our EV client to receive future funding from the government, who is helping with the transition to EV.
In order to achieve the EV client’s aim for compliance and interoperability, a testing strategy was implemented. This included the unit, integration, contract and end-to-end tests, which provided the test coverage needed. While unit, integration and contract tests gave faster feedback and were less costly to write and maintain, they don’t cover the full interaction between services in the client’s EV system. That’s why, in the rest of this section, we focus on the implementation of end-to-end tests (e2e).
In the EV industry, EVs and charge stations are often tested physically. Even though the hardware testing is essential for validation of software, this will be costly to implement and harder to maintain and fix if issues are found from the physical hardware. This is where we need to get validation of the software as early as possible to deal with any issues. Using automated e2e tests helped us to achieve the aim of dealing with issues earlier as part of our continuous delivery to the EV client.
For our e2e tests, we used local docker containers to test the flow of the EV system as a whole. The e2e tests were automated to emulate the communication between the charge station and the CSMS via the OCPP application. This approach helped us shorten the feedback loop, as the automated tests focused on testing the features we were building as part of the OCPP application.
The following two examples demonstrate use of our e2e automation test flow in the client’s EV system:
Example 1 - Charge Station Operator sets the charge station to “Inoperative”
The diagram above shows the regular use of a CSMS by a station operator. In this case, the e2e test framework behaves like the CSMS operator (or Charge Station Operator) to set the charge station’s availability as “Inoperative”. This is most commonly used when there’s a fault with the station or a supporting system.
Once the charge station accepts the operation, it will return a message to the CSMS simulator via the OCPP app. The CSMS simulator will forward the message to the e2e test framework, where it is then validated by comparing it to an actual OCPP response.
Example 2 - EV driver starts to charge their vehicle
Here we can see a simulation of an EV driver connecting their vehicle to a charging station with plug and charge capabilities. In this case, the e2e test framework behaves like the EV driver. The charge station acknowledges the vehicle has been plugged in and communicates that message to the CSMS simulator via the OCPP app. The e2e test framework then validates the message received from the CSMS simulator to ensure it is correct and as expected.
Testing for OCPP compliance
During the same engagement, we also had to ensure that all OCPP messages were compliant with the OCPP 2.0.1 specifications. To achieve this, we validated them using a compliance testing tool that is provided by the Open Charge Alliance (OCA). This tool is known as OCTT. We can use this tool to validate against the System Under Test (SUT), which can be either the implementation of a CSMS or the Charge Station.
For our engagement with the EV client, we used OCTT to assess the charging station infrastructure in accordance with the OCPP 2.0.1. The following diagram illustrates one OCTT test case (TC_G_03_CS - Change Availability EVSE - Operative to inoperative) that was used to assess our EV client’s charging station infrastructure:
We also used OCTT to assess the implementation of our open source Charge Station Management System (CSMS) reference software (MaEVe) in accordance with OCPP 2.0.1. This diagram illustrates OCTT’s test case (TC_F_20_CSMS - Trigger message - Heartbeat) that was used to assess the CSMS:
Once the SUT has been fully assessed by OCTT, an OCPP certification is granted, and the organization can pursue any relevant non-mandatory certifications such as Advanced Security, Smart Charging or ISO-15118 support.
Efficient, reliable EV ecosystems demand robust testing
To operate efficiently and reliably, communication must flow seamlessly between every actor, touchpoint and protocol in the EV ecosystem. When you’re building and developing that ecosystem, the only way to ensure it will operate in the way you intend is to rigorously test every connection.
End-to-end testing makes sure that the various elements that make up the EV ecosystem can all work together seamlessly, and ensure that as those elements evolve — through software updates for example — communication remains fast, consistent and compliant.
To learn more about our ongoing work across the EV space, and discover how we’re helping organizations build robust EV ecosystems and components, visit our blog page.
Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.
