This presentation makes the case for BDD in general and focuses on its use within embedded software development. Using the Cucumber gem, I will demonstrate how to use feature files in the context of working with embedded hardware projects, and explain how to extend the framework using the Wire protocol to allow integration tests to run in-situ, which greatly enhances testing coverage compared to PC-based testing using emulators. I also cover the notion of a SpecFlow gateway, with which one can achieve end-to-end testing with a variety of devices as an orchestration mechanism for broader tests.

1. EMBEDDED SOFTWARE
DEVELOPMENT USING BDD
Itamar Hassin
September 2014

2. THIS TALK COVERS
•Embedded development challenges
!
•The case for BDD
!
•Simulating the target
!
•Accessing the target remotely/in-situ
!
•Orchestrating access to multiple targets

3. EMBEDDED DEVELOPMENT CHALLENGES
•A simulator rarely available or not fully functional
!
•Remote device verification
!
•Complex state-machines
!
•Test coverage often limited to unit testing

4. THE CASE FOR CUCUMBER
•Direct mapping from user story acceptance
criteria
!
•Living documentation, unified view of the product
!
•Helps defines ‘done’: Code is tested and validated
!
•BDD promotes lean code & emergent design
!
•Authored by the team: BAs/QA/Devs

5. CLOSER TO THE SOURCE

6. PREMISE FOR AUTOMATION
Programatically validate that code solves the
problem by articulating behaviour in
machine-readable form.

7. CUCUMBER FOR EMBEDDED!

8. WRITE ONCE, USE THRICE
Feature: Alarm assured to appear in quiet mode
!
Scenario: Pressure alarm
Given device is in quiet mode
When pressure sensor is disconnected
Then a silent alarm will appear

9. IMPLEMENT STEPS
Given(/Given device is in quiet mode $/) do
@device.set_quiet_mode(1)
end

10. IMPLEMENT A SIMULATOR
class Device
def set_quiet_mode(flag)
if (flag)
mode |= QUIET_MODE
else
mode &= ~QUIET_MODE
end
update_driver(mode)
end
end

11. VALIDATE UNDER SIMULATOR
Scenario: Pressure alarm
Given device is in quiet mode
When pressure sensor is disconnected
Then a silent alarm will appear
!
5 scenarios (5 passed)
26 steps (26 passed)
0m0.052s

12. WHEN SIMULATION IS NOT
ENOUGH

13. THE “WIRE”
•When your system does not have native support
•When you want a lean, portable implementation

14. SIMPLIFIED WIRE PROTOCOL

15. WIRE IMPLEMENTATION
BLUEPRINT
•TCP/IP loop managing Cucumber protocol
•Function table for API invocation
•API implementation returning status to Cucumber

16. HOOKING CUCUMBER TO
LISTENER
features/step_definitions/cucumber.wire
host: device
port: 3901

17. LISTENER TCP/IP LOOP
while(fgets(buff, sizeof(buff), rStream))
{
respond_to_cucumber(wStream, buff);
}

18. WIRE HANDLER
void respond_to_cucumber(FILE* stream, char* msg)
{
if (MSG_TYPE_IS(msg, "step_matches"))
respond_to_step_matches(stream, msg);
else if (MSG_TYPE_IS(msg, "invoke"))
respond_to_invoke(stream, msg);
else if (MSG_TYPE_IS(msg, "begin_scenario"))
respond_to_begin(stream, msg);
else if (MSG_TYPE_IS(msg, "end_scenario"))
respond_to_end(stream, msg);
else
respond_success(stream);
}

19. FUNCTION TABLE
stepdef_t stepdefs[] = {
{ "device is in quiet mode”, set_quiet_mode }
};

20. INVOKING API FUNCTIONS
void respond_to_invoke(…)
{
int id = get_function_id(msg);
!
stepdefs[id].callback(arg_val) ?
failure(stream) : success(stream);
}

21. TARGET API
int set_quiet_mode(const char *arg)
{
context->requested_mode = atoi(arg);
context->quiet_mode |= context->requested_mode;
return(update_driver(context));
}

22. IMPLEMENTATION STACK

23. WORKING WITH CUCUMBER
•Decide on a strategy (off-board, on-board)
•Get appropriate toolchain (cross compiler, linker)
•Implement and port Wire to target
•Run the feature files
•fail/implement/pass/refactor/repeat

24. USAGE PATTERNS
Off-Board
• Framework on PC
• Listener on PC
• Proxy API on PC
• Network calls to Target API
!
In-Situ
• Framework on PC
• Listener on Target
• API calls on Target Progression

25. WORKING AT A SAFE
DISTANCE

26. OFF-BOARD
Cucumber
running on
PC
Wire running
on PC
Target API
running on
PC
C-implementation Network
Target
• + Target untouched
• - All API’s must be exposed;
low-fidelity; many moving
parts;

27. UP CLOSE AND PERSONAL

28. IN-SITU
Framework
running on
PC
Cucumber-
Wire running
on Target
Target API
Network C-implementation
• + high-fidelity, API’s not exposed
• - Server part of codebase

29. COLLABORATIVE
ENVIRONMENT

30. GATEWAY
!
•Acts as an end-to-end test orchestrator
!
•Switchboard events across heterogeneous devices

31. COLLABORATIVE
END-TO-END TESTING
Framework
running on
PC
Cucumber-
Wire running
on Target
C-implementation
Targets
Native
Wire
Collaboration

32. GATEWAY ARCHITECTURE
SpecFlow
Target B
Proxies
A1
B1
Hardware
Serial
Wire
Cucumber Target A
Behave

33. END-TO-END FEATURES
Feature: Alarm assured to appear in quiet mode
!
Scenario: Pressure alarm
Given device is in quiet mode
When pressure sensor is disconnected
Then a silent alarm will appear

34. GATEWAY STEPS
public class QuietModeSteps
{
SignalSimulator signalSimulator = new SignalSimulator();
MedicalDevice medicalDevice = new MedicalDevice(“192.168.1.1”, 3901);
!
[Given(@"device is quiet mode")]
public void GivenDeviceIsQuietMode()
{
NUnit.Framework.Assert.IsTrue(medicalDevice.SetQuietMode());
}
!
[When(@“pressure sensor is disconnected")]
public void GivenPressureSensorIsDisconnected()
{
NUnit.Framework.Assert.IsTrue(signalSimulator.SetPressure(off));
}
}

35. GATEWAY PROXIES
class MedicalDevice
{
protected Wire wire;
!
public MedicalDevice(string ipAddress, int port)
{
myAddress = ipAddress;
wire = new Wire(myAddress, port);
wire.Open();
}
!
public bool SetQuietMode()
{
wire.Send("["step_matches",{"name_to_match":"set quiet mode on"}]n");
wire.Send("["invoke",{"id":"7","args":["on"]}]n");
return(wire.Ack());
}
}

36. EMULATING WIRE
public class Wire
{
public int Open()
{
client = new TcpClient(myAddress, myPort);
stream = client.GetStream();
return(Send(“[”begin_scenario"]n"));
}
!
public int Close()
{
stream = client.GetStream();
Send("["end_scenario"]n");
return(client.Close());
}
}

37. SPECFLOW TO WIRE
SpecFlow
int SetQuietMode(“on”) {}
Wire
Proxies
A1
Target
TCP
Given … quiet mode
Wire
Match:
“set quiet’(on|off)’”
Invoke:
idx:0, params: “on”
A
int set_quiet(char* state){}

38. MAINTENANCE
CONSIDERATIONS
Exists in proxy? Write wrappers
Exists in Wire?
No
No
Yes
Function table entry
Yes
API Exists? No Implement API
Yes
Use in feature/step files

39. COMPLIANCE
CONSIDERATIONS
•Security - Anyone can connect to Wire!
•Regulation may not allow non-application code on
a production system
Shut down the wire thread in production

40. LESSONS LEARNED
Threads & Target Dual Vocabulary
Architecture
Threading

41. REFERENCES
•Specification by example
•The Cucumber Book
•Cucumber Recipes
Photo Credits:
@history_pics/@historyinpics
Jim Reese#Wikipedia
National Library of Australia
•SpecFlow

42. THANK YOU!
@itababy
www.in-context.com