fbpx

Independent, impartial advice for car buyers and car owners

How our Expert Rating scores are calculated

Our Expert Rating index is now the gold standard for rating and ranking new cars in the UK

There are dozens and dozens of automotive websites in the UK, and most of them include reviews of new cars. All up, there are tens of thousands of UK car reviews on the internet.

So how do you decide whose opinion to believe when you’re looking for a new car? Is Auto Express better than Autocar? How does the Telegraph compare to the Sunday Times? Or what about The Car Expert and Honest John? All of their reviews are written by experienced journalists, but they’re still all human beings with their own preferences and biases.

Not only that, but every website has its own way of measuring and scoring cars, so a review of 7 out of 10 may be above average from one site and below average from another site. If you’re only relying on one source, you’re not getting the full picture.

Fortunately, we’re here to help with our unique new Expert Rating index.

What is an Expert Rating?

The Expert Rating score for a car is an aggregated rating made up of a number of individual review scores from up to 25 different sources. This can be as few as 12 reviews and in some cases is more than 60 (as some cars have multiple variants of engines, body styles and trim levels).

The world’s most famous review aggregator website is Rotten Tomatoes, which compiles reviews for movies and TV shows. It then rates them on the number of positive reviews vs negative reviews, but there are many different methods of aggregating scores.

The Car Expert is the first website to successfully apply review aggregating to new car reviews. Our system uses a “weighted average” of all the reviews of a particular car published by the 25 sites we include in our index. This is a similar process to sites like Metacritic (another film/TV/music review aggregator site), although we have our own proprietary algorithm developed specifically for new car reviews.

A weighted average is more complicated than a straight average (which is what we used to use) because there are additional factors we take into consideration.

We factor in the age of each review, so that newer reviews carry more weight than older reviews, and we normalise the scores from each title to account for variations in the scoring systems they each use.

It’s taken us a couple of years to develop and test the calculations to make sure they work, which is why we’ve only recently shifted from a simple average to our new weighted average system.

It’s no understatement to say that this is now the gold standard for rating and ranking new cars in the UK.

Our Expert Rating index is now the gold standard for rating and ranking new cars in the UK.

How can you justify that claim?

Rather than one person’s opinion or one website’s view on what makes a car good or bad, we compile and analyse new car reviews from 25 of the top UK motoring websites. Then we apply a complex set of calculations to bring you the definitive rating for every new car.

Our advanced Expert Rating algorithm factors in the age of the review and the scoring methods used by all 25 different sites. It constantly recalculates and updates the Expert Rating score for every single car in real time. The scores can shift every day and every time a new review is added to the database.

For car buyers, it’s an easy-to-use guide to all the key information on each new car. You can see, at a glance, what each website has said about a car, and click on a link to visit that site and read the full review. We also include complete Euro NCAP safety ratings, Green NCAP eco ratings and a list of any awards the car has won.

So far, we have published nearly 300 new car ratings and we’re adding more each week. Our database contains more than 8,000 individual reviews and is growing daily. We currently track 25 automotive titles and we’re planning to grow that to 30 in coming months.

FAQs

How do you calculate the Expert Ratings?

The Expert Rating score is an aggregated rating made up of a number of individual review scores. Unless it’s a very niche model, we usually require a minimum of 12 scored reviews before we include a car in our Expert Rating index. Some cars have more than 60 reviews used in our calculations.

Our system uses a “weighted average” of all the reviews of a particular car published by the 25 sites we include in our index.
We factor in the age of each review (newer reviews carry more weight than older reviews) and we normalise the scores from each publication based to account for variations in how they each score their reviews.

It’s taken us a couple of years to develop and test the calculations to make sure they work, which is why we’ve only recently shifted from a simple average to our new weighted average system.

So how exactly are the scores from each title weighted?

Sorry, that information is classified Ultra Top Secret and is above your security clearance.

What we can tell you is that it’s a complex mathematical algorithm applied universally to all reviews. We don’t manually alter any individual review scores, but every review is put through our super-secret algorithm.

What about car reviews that don’t provide a score?

Some titles review a lot of cars but they don’t provide scores (like the Daily Mail and the Sun). Other titles won’t score reviews for a short test and will only score vehicles on longer reviews. These reviews are still added to the database so you can find them and read them, but they don’t count towards the Expert Rating score.

Unlike some aggregator sites (Metacritic is one example), we don’t manually alter scores up or down, or create scores for unscored reviews. That works for their system, but we don’t think it’s the right solution for us.

That car’s score has changed twice in the last month. Why is that?

It’s perfectly normal for Expert Rating scores to shift up or down over time, and there are a few reasons for that happening.

Firstly, we are constantly adding new reviews to our database. For a new model launch, the big sites (Auto Express, Autocar et al) tend to get first crack, then the secondary sites. Finally, niche sites and regional titles get to review the cars. This process can take months. Then there will be additional versions (new engines, new trim levels) and facelifts, so the whole process repeats itself again and again.

Secondly, some sources will adjust their reviews scores over time. This may be as cars are re-evaluated against newer rivals, or it may be for a specific reason. For example, the 2021 Dacia Sandero was originally given quite high scores from most sources – right up until Euro NCAP crash-tested the Sandero and gave it a very poor two-star score. Several sites immediately downgraded their scores accordingly (and, in one embarrassing case, had to revoke a ‘Car of the Year’ award…).

Thirdly, the reviews are weighted for age. Older reviews carry less weight than newer reviews in the overall score.

Finally, our advanced algorithm automatically accounts for any changes in scoring scales or methodology from any of our sources.

Which publications contribute towards the Expert Ratings?

We started out tracking 12 sites, concentrating on the largest and best-known automotive publications that review the most cars. Over the last two years, we have broadened that out to a current total of 25 carefully selected sites, including national newspaper sites that have substantial and regular motoring content. We have also added a few specialist titles that target specific areas of the new car market (green cars, fleet buyers, performance cars).

As of August 2021, the list is:
Auto Express, Auto Trader, Autocar, Business Car, Car, Car Keys, Carbuyer, Carwow, Company Car Today, Daily Mail, Daily Mirror, Eurekar, Evo, Fleetworld, Green Car Guide, Honest John, Motors.co.uk, Parkers, The Car Expert, The Sun, The Sunday Times, The Telegraph, Top Gear, What Car? and Which?

Why have you selected those sites?

We have a range of criteria for inclusion, but mainly we’re looking at the quality and quantity of reviews. We also consider the variety of cars reviewed by each publication and how the reviews are scored. We only take websites into account, rather than printed titles, so we can always link back to the original review source.

We also review our existing sites periodically and could remove some sites from the index if we don’t think they are contributing sufficiently to warrant their ongoing inclusion. By Christmas 2021, we hope to have added a few more sites to bring us to a total of 30. But any future expansion is dependent on review quality rather than simply adding more sources for the sake of it.

We only use UK sites in our Expert Rating data. A car’s pricing and specification can vary dramatically around the world, so a review from a US or Australian website won’t necessarily reflect that car’s position in the UK market.

Does anyone else use your rating technology?

As of August 2021, our aggregated ratings are available to licence. The data is already being used by Leasing.com, one of the UK’s leading leasing comparison sites. Leasing.com’s ratings are powered by the same review data as our own Expert Ratings, but we run them through a bespoke algorithm to provide a unique set of scores.

If you would like to enquire about licensing our ratings, please email editor@thecarexpert.co.uk.

How often are the Expert Ratings updated?

We’re updating information in the Expert Rating section of the site daily, although that doesn’t mean we update every car every day. On average, each car will be updated about once a month so the information is about as current as you’ll see anywhere.

How do you decide which cars to add to the index?

Ultimately, we want to include every new car on sale and we’re working as fast as we can to get there.

Broadly speaking, we’ve tried to get all of the most popular cars included as a priority. So that means there are more Fords and Fiats than Ferraris, although we’ve made sure that we have at least one or two cars from every major car manufacturer.

How do you separate different versions of the same car?

It can get tricky deciding whether to lump different versions of a model together or rate them separately. We look at this on a case-by-case basis.

For example, the Audi A4 has a multitude of different variants. We include the saloon, Avant (estate), A4 allroad and S4 versions all under the one general umbrella of “Audi A4”. But the Audi RS 4 is considered separately because it’s a very different vehicle.

Similarly, we consider electric models separately from petrol/diesel/hybrid models. A good example is the Vauxhall Corsa (petrol/diesel) and the Vauxhall Corsa-e (electric).

Do you factor in Euro NCAP scores?

No, they are separate. We only use the overall review scores from each title. In many cases, these scores specifically take Euro NCAP safety ratings into account, so the Euro NCAP rating is indirectly included anyway. For example, Which? will heavily mark down any car with a Euro NCAP score of three stars or worse.

Do you include owners’ reviews from any of the sites?

No, we don’t count user reviews even though some of the sites we track display owners’ ratings or carry out owner surveys (like the annual Auto Express Driver Power survey).

Owner reviews have some good points, but they can also be problematic. Most of the time, owner surveys are completed by people who’ve just bought a new car and are still in love with it, or by those who have had a bit of a nightmare and want to vent about it.
This tends to create a very polarised data set of very good and very bad reviews, rather than balanced and objective opinions that encompass a range of scores. You see the same thing across the internet in almost all situations – just look at Amazon for a good example.

Owner reviews are also much harder to verify and easier for PR departments (or salespeople, Russian bots, Extinction Rebellion or MI6) to influence, whereas reviews from recognised media sources and journalists are more reliable and trustworthy.

We are investigating ways of aggregating different user reviews to create an overall Owner Rating score to sit alongside the Expert Rating score, but so far we have found the data to be too sporadic and unreliable to provide a valuable score that readers can trust.

Can you add my site to your Expert Ratings?

We are currently evaluating additional websites for inclusion in our index. We have a range of criteria for inclusion, but mainly we’re looking at the quality and quantity of reviews, the variety of cars reviewed and how the reviews are scored. We only take websites into account, rather than printed titles, so we can always link back to the original review source.

We also review our existing sites periodically and could remove some sites from the index if we don’t think they are contributing sufficiently to warrant their ongoing inclusion.

If you would like your site to be considered for inclusion in our index, please email editor@thecarexpert.co.uk.

How to read our Expert Ratings information

Expert Rating

Our unique Expert Rating is calculated by compiling new car review scores from across all the top UK automotive media titles, rather than just our own opinions. This means you get a balanced view of multiple opinions from up to 25 different sources, with links back to all the original articles so you can read them for yourself.

Safety Rating

We list the full Euro NCAP safety ratings for each model, rather than just the headline score. This includes a complete breakdown of scores for adults, children, pedestrians and assist systems, so you can see the information that matters most to you.

Eco Rating

We include the new Green NCAP eco ratings for each car where they are available, including air quality and energy efficiency ratings for each model and engine tested. This programme only started in February 2019 and was paused in 2020 as a result of the Covid pandemic, so there are relatively few models listed so far, but this will grow quickly.

Security Rating

We have security ratings for several new cars as tested by Thatcham Research, which assess the theft risk of new cars. This includes theft by relay attack for keyless entry/keyless start systems. The programme started in 2019 but has been discontinued as of 2021, so very few cars currently have a security rating.

Last updated: 12 August 2021

Stuart Massonhttps://www.thecarexpert.co.uk/
Stuart is the Editorial Director of our suite of sites: The Car Expert, The Van Expert and The Truck Expert. Originally from Australia, Stuart has had a passion for cars and the automotive industry for over thirty years. He spent a decade in automotive retail, and now works tirelessly to help car buyers by providing independent and impartial advice.

What are your thoughts? Let us know below.

Best car rankings

The Car Expert has the ultimate ranking of the best and worst new cars on sale, based on scores from 25 of the top UK motoring websites.

We’ll be adding more cars and more lists all the time, so keep checking back for the latest results.