Página cargando

SeeQuestor

Solve crimes faster with privacy at the core

SeeQuestor demo presentation Faraday Theatre, Royal Institution (UK)
Duration:36 months
Budget:£ 2.5M
Year:2016-2018
Client:SeeQuestor Ltd (UK)
#machinelearning #video #crime #forensic

Solve crimes faster with privacy at the core

A major crime or incident has occurred. The criminal is at large. Urgent action is needed and a judge has allowed for the use of surveillance camera recorded footage but forensic analysts cry in despair.

This is more or less the story that Henry Hyde-Thomson, physicist turned into entrepreneur turned into serial entrepreneur turned into CEO of SeeQuestor told us when we met in London in late 2015. In his office at The Royal Institution he asked us "Can Kaleidos build a machine learning powered video forensic analysis platform for law enforcement agencies?". "Has this been developed before?", "No, this would be the first of its kind" he replied. "Sure! As long as there is sound science backing the technology and we adhere to a strict ethics protocol, we're in! Count on us!"

But what is SeeQuestor exactly and what needs does it satisfy?

seequestor awards

The challenge

Contrary to public belief, law enforcement agencies struggle to make use of the hundreds or thousands of video footage available after a crime. We're talking homicide, abduction, rape, terrorist attack and the likes. Even if a judge is quick to allow for surveillance camera material to be collected by the police, ending up with tons of material is more a curse than a blessing.

TV will typically depict supersmart software reconstructing scenes, depixelating faces or vehicle registration plates and finding the criminal in 25 min plus commercials. The reality, believe it or not, is much harsher than that. Forensic analysts have to actually watch all that footage (often ranging back several days or weeks), annotate it and try to use their brain gray matter to solve the case, whether that means finding the abducted person, preventing an even worse incident or catching the main suspect.

You can skip to the final results watching this mind-blowing live demo at the Royal Institution in London



Data shows that law enforcement agencies almost always give up reviewing all that material and instead rely on other techniques. But what if a platform could perform a complete analysis of thousands of hours of footage of dozens of different cameras, view angles, locations and dates in a matter of minutes? That would mean faster case solving and, in many cases, solving it at all.

a crowded airport terminal

The Ethics kicks in hard

Now, who doesn't want to help building a platform that can prevent a rapist becoming a serial rapist or catch a terrorist shortly after a carnage happened. But there is a big BUT. You shouldn't use the platform for nefarious purposes, whether they are backed by the country or the organisation or it's a rogue officer that is abusing the system.

SeeQuestor was very keen to add a Human Rights clause to our contract "as long as no party has any superior moral standing" as SeeQuestor Chairman Tristram Riley-Smith would like to say.

content of the human rights clause:
Both SQL and Kaleidos are committed to ensuring that the design and development of the Software will comply with the principles of Article 8 of the European Convention on Human Rights (“Right to respect for private and family life”). If either Kaleidos or SQL feel these principles are being infringed by the design and development of the Software, and if Kaleidos and SQL cannot agree a solution within a 2 week timeframe, either party will seek the counsel of a regulatory and/or civil liberties group (such as, in the UK, the Surveillance Camera Commissioner or Big Brother Watch). If either party continues to feel its ethical principles are being compromised by involvement with the design and development of the Software (despite either reassurances being given or remedies being proposed and accepted), that party will be free to terminate its engagement with the Project within 30 days, and the other party reserves the right to seek appropriate remedies in accordance with clause 15

They also employed the reputed Professor Tom Sorell as their Chief Ethics Officer. He would establish the criteria for an ethics-driven sales strategy. We enrolled Big Brother Watch and other civil-rights organisations in the UK to make sure we were building this "the right way" and they assured us that "this is exactly the approach all other initiatives should follow, kudos for this".

Nevertheless, one Kaleidos team member opted for conscientious objection because she didn't trust future uses of the platforms outside our reach.

people from Kaleidos and Seequestor working together

The platform

Over the course of 2 years we built a platform that would harness the power of machine learning to extract thousands of full-body signatures out of hundreds of different media formats. It will allow forensic analysts to create cases, ingest thousands of hours, automatically process everything and get motion detection, face detection and body detection profiles for all of them.

That way they could do reidentification, which is not the same as recognition. Recognition is linked to the question, "Who is this person?" while reidentification is linked to the question "Where is this person found elsewhere in my footage?". And that question got asked hundreds of times per case. It was amazing to see forensic analysts suddenly evolve into superior Sherlock Holmes thanks to a platform meant to give instant feedback to all their amazing ideas (some of them truly longshots that could now pay off).

We had designed the system with the help of Scotland Yard, British Transportation Police, FBI and Queensland Police. It was no wonder that users felt it was exactly what they needed. What some of them didn't know is that behind that web platform there was a mini supercomputer that dwarfed IBM's Blue Gene, crunching data thanks to Queen Mary University team led by Professor Sean Gong.

Dashboard of the app with a zoom detail of a real crime

I rest my case

When you are designing a mission critical system that cannot fail and cannot mislead, there comes a point when you need to prove that it actually works. And you need to prove it in front of the most sceptical audience, there are no shortcuts. If it doesn't work REALLY well, it's just a gigantic piece of pretentious software rubbish.

We went through many double blind experiments where a police force would ask an external forensic analysts to solve a past case (criminal already serving sentence, no pressure there) with the same input they had had at the time. There was one such double blind test where SeeQuestor helped solve the case in 2 hours. They were stammering when they told us it had taken them close to 300 hours over the course of months. To make matters worse, it was a sexual predator case.

One after another, police investigators enjoyed Eureka moments using SeeQuestor and success stories started to mount up. Only a few made it to the news but the gratitude from these people was never-ending and sometimes understandably emotional.

detail of the app with a zoom of a real crime

The handover

Kaleidos has a clear policy not to continue building a platform much beyond its market launch since we believe companies should have in-house teams for that. SeeQuestor wasn't quite ready for that departure so they challenged us again with their next big idea; 3D scene reconstruction out of simple 2D video footage. But that's another story. SeeQuestor finally got their hands on the massive yet sustainable platform we had built. We helped them recruit their IT team so they could continue evolving the platform we had developed for them and wished them good luck!

SeeQuestor project was a unique combination of trustworthy and ethics-aware individuals, sound science and a very challenging goal. In hindsight, I'm not sure we would have embarked ourselves on that project without making some changes first. It helped us shape Kaleidos' ethics committee and conscientious objection, and also prove that you can have an IT partner that can lead product development when so much is at stake and so little prior art is available.

Kaleidos is going to be such a tough act to follow...
Henry Hyde-Thomson, SeeQuestor CEO
Visit SeeQuestor Website
We learned that...We had to deal with a complex blend of Academia, Law Enforcement Agencies and Startup mindset. Agile worked!
One key takeaway was...The platform evolved based on real-life case usage feedback. It was a virtuous cycle that led to a game changer.
Do you like what you see?Let’s talk about your project get in touch
Main officeLa Botánica 4, 1º, 28028 Madrid, Spain
Kaleidos Web SVG sprite