Benfromparis's comments

DataDome | NYC / Paris | Full Time | Onsite or Remote https://datadome.co https://docs.datadome.co/

Bot hunters is a full time job! With our SaaS cybersecurity technology, we detect and manage non-human traffic in real-time to protect our clients’ websites and APIs.

Named as a Strong Performer by Forrester, DataDome is a growing cybersecurity pioneering in bot detection and protection for content publishers, eCommerce websites and advertisers across the world. We handle and protect more than 7 billions daily below 3ms (99p).

Our stack:

• Stream data processing: Kafka / Flink / Elasticsearch

• Real time detection engine: Java / Scala

• Webservers modules (nginx,haproxy…): C++ / C#, nodeJS, lua

• Infrastructure: 1000VM (AWS/GCP/Azure/baremetal)

• Dashboard: Angular7 / symfony3

We’re expanding our team and are looking to hire:

• Cyber Security Analyst, Data Analyst

• Senior JavaScript Engineer, Senior Backend Engineer

• Technical Support Engineer

• DevOps/SRE Engineer

Join us: https://www.datadome.co

DataDome | NYC / Paris | Full Time | Onsite or Remote https://datadome.co https://docs.datadome.co/ Bot hunters is a full time job! With our SaaS cybersecurity technology, we detect and manage non-human traffic in real-time to protect our clients’ websites and APIs. Named as a Strong Performer by Forrester, DataDome is a growing cybersecurity pioneering in bot detection and protection for content publishers, eCommerce websites and advertisers across the world. We handle and protect more than 7 billions daily below 3ms (99p).

Our stack:

• Stream data processing: Kafka / Flink / Elasticsearch

• Real time detection engine: Java / Scala

• Webservers modules (nginx,haproxy…): C++ / C#, nodeJS, lua

• Infrastructure: 1000VM (AWS/Azure/baremetal)

• Dashboard: Angular7 / symfony3

We’re expanding our team and are looking to hire:

• Senior JavaScript Engineer, Senior Engineering Manager

• Technical Support Engineer

• DevOps Engineer, DevOps Engineer Manager

Join us: https://www.welcometothejungle.com/en/companies/datadome/job...

404 error

"200k documents per second" would be less catchy you are right, but it also doesn't reflect the complexity to handle that throughput. Most of the benchmark we can see around like "1m document per second" are using small documents in POC environement. In our setup, each of the 250 fields are store and indexed in ES, making it CPU and I/O intensive, in a production environment.

Good catch! We will add it in the article

I'm a bit surprise that a well-known company publicly assume to crawler and workaround bot protection solutions.

Crawling is against Booking CGU : "Our Trip Service is made available for personal and non-commercial use only. Therefore, you are not allowed to re-sell, deep-link, use, copy, monitor (e.g. spider, scrape), display, download or reproduce any content or information, software, reservations, tickets, products or services available on our Platform for any commercial or competitive activity or purpose."

DataDome | NYC / Paris | Full Time | Onsite or Remote https://datadome.co https://docs.datadome.co/ Bot hunters is a full time job! With our SaaS cybersecurity technology, we detect and manage non-human traffic in real-time to protect our clients’ websites and APIs.

Named as a Strong Performer by Forrester, DataDome is a growing cybersecurity pioneering in bot detection and protection for content publishers, eCommerce websites and advertisers across the world. We handle and protect more than 4 billions daily below 3ms (99p).

Our stack:

• Stream data processing: Kafka / Flink / Elasticsearch

• Real time detection engine: Java / Scala

• Webservers modules (nginx,haproxy…): C++ / C#, nodeJS, lua

• Infrastructure: 100VM (AWS/Azure/baremetal)

• Dashboard: Angular6 / symfony3

We’re expanding our team and are looking to hire:

• Senior Software Engineer : Java, Scala, C++

• Data Engineer / Data Scientist : Scala, Java, Python

• Support & Onboarding Manager

• FullStack Web Engineer : Angular5 / symfony3

Join us: [email protected]

DataDome | NYC / Paris | Full Time | Onsite or Remote https://datadome.co https://docs.datadome.co/

Bot hunters is a full time job! With our proprietary SaaS cybersecurity technology, we detect and manage non-human traffic in real-time to protect our clients’ websites and APIs.

Named as a Strong Performer by Forrester, DataDome is a growing cybersecurity pioneering in bot detection and protection for content publishers, eCommerce websites and advertisers across the world. We handle and protect more than 4 billions daily below 3ms (99p).

Our stack:

• Stream data processing: Kafka / Flink / Elasticsearch

• Real time detection engine: Java / Scala

• Webservers modules (nginx,haproxy…): C++ / C#, nodeJS, lua

• Infrastructure: 100VM (AWS/Azure/baremetal)

• Dashboard: Angular6 / symfony3

We’re expanding our team and are looking to hire:

• Senior Software Engineer : Java, Scala, C++

• Data Engineer / Data Scientist : Scala, Java, Python

• Support & Onboarding Manager

• FullStack Web Engineer : Angular5 / symfony3

Join us: [email protected]

DataDome | Paris, France | Full Time | Onsite

https://datadome.co https://docs.datadome.co/docs

Bot hunters is a full time job! With our proprietary SaaS cybersecurity technology, we detect and manage non-human traffic in real-time to protect our clients’ websites and APIs.

DataDome is a growing French startup pioneering in bot detection and protection for content publishers, eCommerce websites and advertisers across the world. We handle and protect more than 2 billions daily below 3ms (99p).

Our stack:

• Stream data processing: Kafka / Flink / Elasticsearch

• Real time detection engine: Java

• Webservers modules (nginx,haproxy…): C++ / C#, nodeJS, lua

• Infrastructure: 100VM (AWS/Azure/baremetal)

• Dashboard: Angular5 / symfony3

We’re expanding our team and are looking to hire:

• Senior Software Engineer : Java, Scala, C++

• Data Engineer / Data Scientist : Scala, Java, Python

• Head Of Engineer

• FullStack Web Engineer : Angular5 / symfony3

Join us: [email protected]

DataDome | Engineer, Data | Paris, France | Full Time | Onsite https://datadome.co https://docs.datadome.co/docs

Bot hunters is a full time job! With our proprietary SaaS cybersecurity technology, we detect and manage non-human traffic in real-time to protect our clients’ websites and APIs.

DataDome is a growing French startup pioneering in bot detection and protection for content publishers, eCommerce websites and advertisers across the world. We handle and protect more than 1.5 billions daily below 3ms (99p).

Our stack:

• Stream data processing: Kafka / Flink / Elasticsearch

• Real time detection engine: Java

• Webservers modules (nginx,haproxy…): C++ / C#, nodeJS, lua

• Infrastructure: 100VM (AWS/Azure/baremetal)

• Dashboard: Angular5 / symfony3

We’re expanding our team and are looking to hire:

• FullStack Web Engineer : Angular5 / symfony3

• Senior Software Engineer : Java, Scala, C++

• Data Engineer / Data Scientist : Scala, Java, Python

• VP Engineer

Join us: [email protected]

DataDome | Paris, France | ONSITE FULL-TIME

https://datadome.co https://docs.datadome.co/docs

Bot hunters is a full time job! With our proprietary SaaS cybersecurity technology, we detect and manage non-human traffic in real-time to protect our clients’ websites and APIs.

DataDome is a growing French startup pioneering in bot detection and protection for content publishers, eCommerce websites and advertisers across the world.

Our stack:

• Stream data processing: Kafka / Flink / Elasticsearch

• Real time detection engine: Java

• Webservers modules (nginx,haproxy…): C++ / C#, nodeJS, lua

• Infrastructure: 100VM (AWS/Azure/baremetal)

• Dashboard: Angular5 / symfony3

Achievement: 500 million hits daily protected below 3ms (99p)

We’re expanding our team and are looking to hire:

• FullStack wEB Engineer : Angular5 / symfony3

• Senior Software Engineer : Java, Scala, C++

• Data Engineer : Scala, Java, Python

• VP Engineer

Join us: [email protected]

Basically, we are using two sort of technics : technical and behavior.

Technical : if the UserAgent claim to be a regular browser (let say Chrome 43) we will check on network level if the client implement http protocol like Chrome 43 usually do and on the JS side if the Javascript render is correct for Chrome. In case it's a real Chrome, we will check if the Browser is controlled by automation Tool.

Behavior : we will check if the path of requests is regular according to the website usage.

Disclaimer: I'm working at https://datadome.co, a bot protection tool.

This module seems to introduce nginx stats close to haproxy native monitoring webpage. I'm wondering what others features are missing in nginx compare to haproxy?

I'm surprised nobody mentioned Laravel. It seems to be one of the most, if not the most, used PHP Framework nowadays?

Of course I cannot go much into details, but we are using behavior detection and Javascript tracking (mouse, scroll, screen...).

Interesting for unprotected websites but it's easy to detect and to block: no valid js, no valid meta header, no valid cookie, suspect behavior...

Selenium is a much "elaborated" solution, but still, can be detected most of the time.

Disclosure: I'm DataDome co-founder. If you want to detect bad bots and scrapers on your website, don't hesitate to try out for free and to share your feedback with us https://datadome.co