Recommended Robustness Tests

Understanding how the SIM and device interact during development can prevent surprises after deployment

Each network is set up a little differently and is a challenge for Global Connectivity. Planning for network heterogeneity is a great way to make sure your devices have the best chance of finding a successful connection. 

Our SIMs are designed with network heterogeneity in mind and follow global telecommunication standards (for example GSMA) to ensure that this is possible. Most cellular modules are designed to meet 3GPP standards to ensure they work with the SIMs built to telecommunication standards.

This should mean SIMs are already network agnostic. However, within these standards, we have found cellular modules differ in the implementation of these standards and can often be optimized for a specific network or SIM type. Often these nuances are only discovered after they are in the field when there is no access to the device.

Therefore, we recommend that you consider conducting the tests laid out in this article to better understand how they will behave once deployed and become difficult to access.

What you will need for these tests:

  • Access to Onomondo (to view the SIM's network-logs)
  • A basic understanding of network-logs (see: How do I read network-logs)
  • 3-5 test SIMs

Recommended tests:

  1. Automatic network selection based on signal strength
  2. Deactivating a SIM
  3. Network-whitelists limited to a single network

Automatic network selection based on signal strength

Why

For quality of service reasons, we offer plug and play SIMs without specified network steering lists (PLMN preference lists). This allows your device to connect to the first stable connection it finds instead of connecting to a SIM defined “preferred” signal which might be weaker.

Having a preferred networks list on the SIM could in theory make the initial selection of a network faster, but it comes at the cost of not selecting a network with high-quality radio signal. 

Not having a good radio signal can lead to a weak connection which could lead to multiple reconnections and unstable data transmission.

Issue: We have seen some modules and pre-configured devices that by default need a preferred network list on the SIM to work.

Test Suggestions:

  1. Put an activated SIM into your device, go to the SIM details page on the platform and click the network-logs tab (NB all Onomondo SIMs are “active” by default). 
  2. Turn on your device and set the APN to onomondo (no caps) and set the device is allowed to use data on roaming networks (only if required). 
  3. Check the SIMs network logs.

Troubleshooting:

  • There is a GSM attachment but no "data" attachment log (sometimes you will "allowed data" log, sometimes only GSM).
    1. The most common reason for this is that the APN and data roaming is not set. Check these setting on the device. If the APN has been set make sure it is all small letters. If it is a preconfigured device (such as an off the shelf gateway), make sure that the default APN is not still being used. Reset the APN and if that still doesn't work, restart the device.
    2. On rare occasions the device throughs errors when trying to establish a PDP context. We recommend looking at the logs of the device and check the status of the errors occuring during pdp context activation. For example, using the AT command AT+CMEE=2
  • There are no network-logs
    Unless there are entries in the signalling-log, the issue is happening before the device attempts to connect to our network when the device is communicating between the Radio Network Using your own AT command Configuration.
    1. Check the technology preference and band selection of the modem. For example, if you are in a country with no NB-IoT connections, but NB-IoT is preferred, it will slow down the initial connection (up to 30 minutes/if at all - depending on the device)
    2. Use the technical AT Command connectivity test guide to see if you can establish a connection using these agnostic commands. These can also help show where the device is struggling to connect. If you are unsure what is wrong, you can contact us with the logs for further insight.
  • There were denials, now we see no new logs
     Denials occur when a specific network is not on the whitelist. However,  if no networks in the area/country are not on the whitelist the device will try them all. Once they are denied you will need to clear the FPLMN list before the device will see try these networks again. You can see more details under the below section Changing the network-whitelist test.
  • Otherwise, you can try adding a PLMN list to a SIM and see if that improves the result. 

Adding a PLMN list to the SIM can improve initial connectivity, but overall will result in lower quality of service, as the device will always select the most preferred network even with a weak signal over a stronger signal that also may be on a whitelist.

Deactivating a SIM while a SIM was online

Why

Sometimes a SIM will drop off a network. One example of this an occur during planned maintenance periods. We recommend, testing how the device responds when it is disconnected to make sure the device is configured to reconnect after being kicked.

Issue: Not configured properly, some devices require a manual restart. In many IoT situations the device is hard/expensive or impossible to reach once deployed.

Test Suggestions

  • While your device is online, Toggle the activation of the SIM to deactivated and then activated again. 
  • The RAN will receive signalling from us to CancelLocation of the device currently connected to their infrastructure. The device should go offline and come online again after short period of time.

Changing a network-whitelist

Why

When a device attempts to connect to a network, it will first request permission to attach to that network. When our network receives this request it checks the network-whitelist that is attached to the SIM. If the network is on the list, it will be allowed to connect. Otherwise, we will deny the request. This can be seen on the network-logs.

When the device is notified of this, it will write the network to the FPLMN list on the SIM. While the network is on the FPLMN list, the device will not attempt to connect to it in automatic selection mode.

Important test: This test also tests how the device records networks to the FPLMN list. A good fail safe for the device to understand how the device writes to the FPLMN list and adding logic to the device to clear the FPLMN list as back up (for example add it to the device boot up or triggered when the device has failed to attach to network because all networks in range are written to the list.

Test Suggestions

  1. On the SIM you are testing add a network-whitelist with limited or no networks in the area. 
  2. If you have used this SIM before for testing, make sure to clear any connection to a current network. This can be done toggling the activation toggle to deactivated and back to activated. This will send a CancelLocation request to the network. This will make sure that the device asks for permission again next times in tries to connect to a network.
  3. Once the device has been denied the networks, change the test whitelist so that it only covers networks that have been denied.
  4. Toggle the activation of the SIM off and on again and see if the device attempts to connect to one of the networks.

Trouble Shooting

  • No new network-logs
    You will need to clear the Forbidden Network (FPLMN) list on the SIM. Once the FPLMN list is cleared, the device should start connecting to the new networks on the whitelst. 
  • No new network-logs after the FPLMN list is cleared
    Modem modules often have a cache including a list of forbidden networks that is cleared when the module/device is reboot.

FPLMN list is a standard to make connections more effective. We also suggest to consider how this could be implemented in production to make your device more agile. One example, is when the module initializing for the first time. Then a reset of the device/module will also clear the FPLMN list.