Remove Exercise Remove Metrics Remove Testing
article thumbnail

Ensuring the Successful Launch of Ads on Netflix

The Netflix TechBlog

In this blog post, we’ll discuss the methods we used to ensure a successful launch, including: How we tested the system Netflix technologies involved Best practices we developed Realistic Test Traffic Netflix traffic ebbs and flows throughout the day in a sinusoidal pattern. Basic with ads was launched worldwide on November 3rd.

Traffic 351
article thumbnail

Migrating Critical Traffic At Scale with No Downtime?—?Part 1

The Netflix TechBlog

The second phase involves migrating the traffic over to the new systems in a manner that mitigates the risk of incidents while continually monitoring and confirming that we are meeting crucial metrics tracked at multiple levels. Replay Solution The replay traffic testing solution comprises two essential components.

Traffic 349
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Interpreting A/B test results: false negatives and power

The Netflix TechBlog

Martin Tingley with Wenjing Zheng , Simon Ejdemyr , Stephanie Lane , and Colin McFarland This is the fourth post in a multi-part series on how Netflix uses A/B tests to inform decisions and continuously innovate on our products. Have a look at Part 1 (Decision Making at Netflix), Part 2 (What is an A/B Test?), Need to catch up?

Testing 213
article thumbnail

Interpreting A/B test results: false positives and statistical significance

The Netflix TechBlog

Martin Tingley with Wenjing Zheng , Simon Ejdemyr , Stephanie Lane , and Colin McFarland This is the third post in a multi-part series on how Netflix uses A/B tests to inform decisions and continuously innovate on our products. Have a look at Part 1 (Decision Making at Netflix) and Part 2 (What is an A/B Test?). Need to catch up?

Testing 237
article thumbnail

Site-Speed Topography

CSS Wizardry

Any time you run a test with WebPageTest, you’ll get this table of different milestones and metrics. Note the bottom row shows me the Standard Deviation of the tests’ results. Higher variance means a less stable metric across pages. With my pen and paper, I’ll make a note of investigating these specifically in my testing.

Speed 328
article thumbnail

Building High-Quality Software

DZone

Saving several hours preparing and doing a design review will cost hundreds (if not thousands of hours) of fixing issues down the road.nn> unit testsnnI don’t believe that I have to say that in 2021, but I have never seen a quality product without unit tests. We live in a time when everybody is irked by manual testing.

Software 245
article thumbnail

Seamlessly Swapping the API backend of the Netflix Android app

The Netflix TechBlog

To prepare ourselves for a big change in the tech stack of our endpoint, we decided to track metrics around the time taken to respond to queries. After some consultation with our backend teams, we determined the most effective way to group these metrics were by UI screen. For the migration, testing was a first-class citizen.

Latency 243