Testing the EV ecosystem

Almost 14 million new electric vehicles (EVs) were registered globally last year, with adoption

continuing to rise throughout 2024. But as the number of EVs on our roads rise, so too does the

importance of a robust, standardized charging ecosystem.

Across complex electric vehicle (EV) ecosystems, hardware, software and functionality must be standardized to enable seamless interaction between diverse actors and touchpoints including:

• EVs

• EV drivers

• Charge Stations

• Charge Station Management Systems (CSMSs)

• Charge Station Operators (CSOs)

• E-mobility service providers (eMSP)

For these actors to seamlessly interact, there are certain communication protocols which must be exchanged. These include, but are not limited to:

• Open Charge Point Protocol (OCPP)

• Open Charge Point Interface (OCPI)

• Vehicle-to-Grid Communication Interface (ISO-15118-20)

The following diagram illustrates an example of how the above actors and the communication protocols correlate:

To enable consistent, compliant and reliable experiences between every actor and across every touchpoint in the EV ecosystem, these communications must be rigorously tested. So, when Thoughtworks began development on its own open source charge station management system (CSMS) reference software, MaEVe, we devised a robust testing strategy to support it.

In the first of the series, we’ll take a close look at how we’ve tested key communications protocols and interactions within MaEVe.

Testing charge station management systems interactions

A CSMS is a platform designed to manage multiple EV charging stations. It’s a centralized tool that enables organizations to:

• Monitor the status of charging stations

• Collect and analyze charging station data

• Authenticate users before they charge

To allow seamless integration within the EV charging infrastructure, a CSMS communicates with individual charging stations through the OCPP created by the Open Charging Alliance. How effectively it does that directly impacts both the user’s experience of a charging station and the efficacy of remote charge station management. This means rigorous testing of OCPP interactions is crucial.

OCPP uses websockets — a protocol that allows bi-directional communication between a client and server over a TCP connection — as its primary transport. When we were building and validating MaEVe, we established a websocket connection to send and receive different messages and responses to test interactions to and from the CSMS. 

We used a websocket client called Websocat to send various messages from the charging station to the CSMS and display the response. The following diagram illustrates the websocket messages based on OCPP 1.6 using Websocat:

Testing and validating that communications are consistently sent and received by the CSMS helps ensure its correct function when it’s deployed. However, it’s also important to check that those communications can be supported at scale. So, in addition to manual connection tests, we also performed load tests to help us understand the limits of our system. In this instance, we used K6 as our load test tool.

Simulating EV and charge station events

As part of the MaEVe project, we also created an EV and charge station simulator that emulates events which occur between them. Having a simulator enables us to understand how the software behaves quicker and simpler. It is also more cost effective as there is no need for an integration with a physical EV and charge station. The simulator connects to the CSMS, and allows us to simulate events such as: 

• Plugging a connector into an EV

• Authenticating vehicles and drivers using an radio-frequency identification (RFID) card or via plug and charge

• Starting a charge

• Stopping a charge

• Unplugging a connector from an EV

To test the CSMS using the EV and charge station simulator, we needed to send Message Queuing Telemetry Transport (MQTT) messages to it. With MQTT, you publish and subscribe to various types of messages which allows you to execute different scenarios and monitor their effects on EVs and charging stations. 

The initial MQTT message exchange for all the EV charge station simulation events was tested manually using the Mosquitto testing tool. The following examples illustrate the MQTT messages sent to our EV charge station simulator using Mosquitto:

As this was the first time we have done this, we tried to test it manually first to ensure the EV/ charge station simulator is working as intended. These tests were then converted into an automated end-to-end (e2e) test suite written in Golang. The e2e automated test is run whenever there is a change to the code and ensures that the tested functionality continues to work as expected as updates are deployed to both EV and charging station software. There is an implementation of the e2e automation tests under the MaEVe open source project.

eMSP App

eMobility Service Providers (eMSPs) usually provide applications to help EV drivers access their services. These apps are a crucial piece of the EV ecosystem and play a pivotal role in customer experiences and journeys.

eMSP applications communicate with a CSMS through the Open Charge Point Interface (OCPI) protocol. There are two crucial flows that we need to test here — the exchange of credentials between the eMSP backend and the CSMS, and the start/stop charging process. 

To exchange credentials between the eMSP and CSMS, we register our eMSP and store a one-time credential token. We can then trigger the eMSP to exchange credentials with the CSMS. When this is successful, the one-time credential will be removed. The diagram below shows the credential exchange flow via OCPI. 

Being able to charge from an eMSP app is also something that we need to test. Here, we combine the test that was covered in the CSMS section above with the ability of our eMSP backend to communicate with the CSMS, as shown below.

The importance of robust testing in the EV ecosystem and part two 

In part one of this blog, we have seen that there are different touch points involved in the EV ecosystem, which must involve rigorous testing. A few examples (such as the interactions between the CSMS and the charge station and the integration of the eMSP app) have helped us to understand the importance of robust testing in the ecosystem. 

In part two, we will explore more on OCPP and its compliance for our recent client project in the e-mobility industry.