vLLM can only run on linux but it’s my personal favorite because of the speed gain when doing batch inference.

What’s actually making this possible is the PimEyes database. It’s insane that there is a facial recognition database that can be accessed like this. I doubt this is anywhere near legal in the EU.

The media keeps concentrating on Meta and the researchers but you can do the same with phone cameras, doorbell cameras, etc.