Not your average Crash Detection and Reporting
By Rokas – Head of Data Science at Sentiance
Product updates
Our high-impact crash model is built to detect crashes where we expect to see considerable damage to the car and potentially require emergency assistance. Typically, a head-on collision at speeds above 35-40kph results in a peak acceleration of 5g or above. Also, we expect that most often airbags are deployed during those crashes and the vehicle is not in a driveable state after the impact i.e., halts within a few meters from the impact location. We have been actively working together with our clients to launch Crash Detection features that will help them bring products quickly to market and get the most value out of it.
This blog post explores the most recent updates to the Crash Detection model that managed to bring false positive rates down by ~5x without sacrificing true positives. We have also extended it to include debug print functionality improving the QA process and helping build trust in our solution. In addition to that, we present a new crash forensics reporting use case – detecting fraudulent claims.
Much lower false positives without sacrificing true positives.
When we started working on AI-driven crash detection with a human-centric approach together with Autoliv we mainly focused on recall – being able to catch as many crashes as we can. It is not an easy problem, since phone sensors are not always reliable. Different phone positions in the car can lead to different signals and noise levels. After successfully developing a solution that can work on-device and capture most crashes near real-time, we focused on bringing the false positive rate down without sacrificing true positives.
Robust GPS fix handling
Proper handling of GPS fixes is vital for Crash Detection solutions. Speed provides a signal that the car was moving fast enough for the crash to require attention (either medical due to the injury or technical due to the car being in a nondriving condition). NHTSA reports that collisions with a delta-v of ≥35km/h have a ~40% chance of resulting in an injury and a ~90% chance that airbags will deploy. By analyzing speeds above 35km/h we are focusing on crashes where people most likely will need help.
Speed inferred from GPS fixes is also used to identify that the car came to a stop after the event. It helps us avoid false identifications arising due to possible phone bumps. For example, accidentally hitting the phone on the car door while getting out, or your phone falling inside a car while driving. After a crash-like peak is detected while traveling at a substantial speed, our solution checks for up to a minute that the car came to a stop.
A challenge while handling GPS fixes is their accuracy and possible lag. In the last quarter, we improved GPS fix handling logic to make sure we have a robust on-device solution that is prone to known GPS fix flukes. We recommend using a sampling rate of at least one GPS fix per 10s. This way we can provide accurate and reactive crash detection results without having a high impact on the battery life.
Signal energy checks
As you will see in the next section we employ a peak detection algorithm followed by a Neural Network to detect crash-like peaks and estimate their confidence. Unfortunately, even though Neural Networks are good at classifying data similar to the one they have seen, they can get easily confused when they face outliers. There is a need for additional logic that eliminates various cases which are exceptions/outliers. For example:
– Phone sensor might be broken and produce invalid readings (usually ending up producing lots of noise)
– The user might be shaking the phone or even hitting it on a hard surface for some reason
To address those cases we have implemented energy checks, which make sure that before and after the crash-like peak the phone does not show high levels of noise. We tuned it so that normal phone use (like phone handling) does not exceed thresholds, but wild shaking, excessive vibrations, drops, etc. will violate the energy check. It takes 10-20s for the smoothed average signal power (used in energy check) to drop below the threshold after high-energy activities. This check is designed in such a way that it will not affect actual crash signals.
Retrained NN using a new base for transfer learning
There is a common myth that neural networks need lots of data to be trained on. There is a powerful and widely used trick of transfer learning, which allows learning efficiently even with a small amount of data. This technique relies on taking an already pre-trained model on a similar problem and slightly updating it for a new problem. Since at Sentiance we have a robust Transport Classifier model made to work on-device in near real-time we used it as a base for transfer learning. It worked really well for Crash Detection, but we have not stopped here and tried to experiment by using the Phone Handling model as a base. You can read more about it Each use case needs its own solution: how Sentiance delivers tailored driving scores.
Turns out that this enabled us not only to shed a hundred Kb from the model size but improve detection accuracy. This is expected since Crash Detection is about peaks and harsh phone movements that are captured better by the Phone Handling model. The sensor part of our Transport Classifier model tends to focus more on vibration patterns, thus its encoding is slightly less useful for Crash Detection problems.
Putting all things together
By improving GPS fix handling logic, retraining the classifier, and adding energy checks we optimized the model to minimize false positives. Clients then don’t need to worry about making too many wrong calls to the emergency services.
For example, with the current expected crash alert rate of less than 50 reports per million car trips, an average user has less than 1 out of 5 chances of getting an alert in 5 years. As with any alarm, it might go off once in a while, but this is a price you pay for having a high identification rate. Based on NHTSA data, 1 million car trips typically lead to ~7 crashes with injuries. Our analysis shows that we are able to confirm ~5 of those correctly. This evaluation assumes that the car before the impact traveled at a speed higher than 35km/h and the phone was not handled during the crash. Also, the accelerometer sampling rate was at least 26Hz and the GPS sampling rate was at least one per 20 seconds.
Expedite QA with the debug print flag.
Although the deep learning-based crash model brings a sophisticated data-driven model to the device, it is not straightforward to validate. Also, with every model improvement, it gets harder to trick it (simulate a crash). We understand the need to test our solution and fast-track our clients’ QA process to help them simulate the crashes during testing. We added a debug print flag.
This functionality exposes the specific reasons why we did not report a crash event despite detecting a high-impact signal in the accelerometer data. Now if you bump your phone on the table you should see that the transport mode is not a vehicle or the speed is too low. We believe this functionality will lead to a much smoother QA process.
Fraudulent claims detection using crash forensics reporting.
Sentiance crash forensics provides you with a detailed report about the detected crashes, listings where they happened, what was the traffic, the weather, the impact, etc. While useful on its own to investigate crashes it can also be used to inspect fraudulent claims.
Fraudulent claims by their very nature will not be detected by the on-device crash model. Similarly crashes that are not in the scope of the definition of the crash model (e.g., driving at 15kph before impact) will also not be detected by the crash model. Hence, no crash report will be automatically generated by our platform. In order to help claims teams with these scenarios, we added an option to request a crash report manually based on the user id and time range.
If you want to know more about the crash forensics report, check our previous blog post – Car crashes in the future: insurers will win on mobile.
Conclusion.
We believe that improved real-time high-severity crash detections coupled with reporting and an easy QA process will help our clients bring their products quickly to market and get the most value out of our products.
Note, that our solution works on any smartphone (Android and iOS) so that your clients can feel secure without owning an iPhone 14.
If you are interested check in Crash Detection contact us for a demo.