When we begin researching and planning for our trip, we go through pretty standard steps - search for accommodations on our usual booking sites, google travel or local blogs for recommendations, ask friends who have visited or lived in those locations, and look up youtube for vlogs that introduce interesting sights. We also find ourselves relying on Google Maps reviews to decide where to stay, eat, or visit.

We spend quite a bit of time researching and planning, and understanding that not everyone enjoys doing so (hopeful you find this blog useful and subscribe). We include opportunities for serendipity to happen, and ask locals for their recommendations whenever we can. Most of our decisions are good enough, some are very good, and a handful are misses.

For example, in our last trip to New York, a local guide recommended a popular eatery for some Asian food. We looked it up on Google Maps - 3000+ reviews with an overall score of 4.4 out of 5 ⭐, looked good, went there, but it turned out to be underwhelming, and not a place we would recommend. However, his other recommendation to take the Staten Island Ferry was a pretty fun thing to do.

It was not just food ratings, accommodation ratings are flawed too.

When we stayed in an Airbnb, where our only major complaint was traffic noise (I am a light sleeper), the host gently reminded us to leave a rating and her message included this:

We get why she sent this. Airbnb’s rating system can penalise her property severely if her ratings go below 4 stars. This reminded me of a WSJ podcast - Airbnb Ratings: Do Five Stars Tell the Whole Story? Since the stars isn’t of much help, the Airbnb rating system doesn’t make it easy for us to filter for the properties that suit our needs. We end up reading through every review to decide if the property was a good fit for us.

Airbnb’s rating system is not the only one that’s flawed, even the “stars” accorded to hotels aren’t consistent. Most of the rating systems other booking sites use don’t work well for BOTH owners and travellers. Owners may find it hard to recover from bad (initial) ratings, or the system don’t enable them to improve their offering.

We need to better system, but this is waaaaaay above our paygrade 😄. We continue to play along this (poor) rating games of awarding the “right” stars because we have no control those systems. But here in Idle Tradventures, we have complete control over how we rate!

How Should We Rate?

We shall rate only 3 things - Accommodations, Food, and Attractions. We initially tried to adopt a quantitative approach, and present it visually using a radar chart. Here is an example:

But there are a number of challenges using this approach. Briefly, some of the challenges (using the example of rating a hotel):

1. How do we choose what variables to include when there are so many to use.

2. We only visit each location once or twice, and so lack sufficient hotel stay experiences to form a base level for scoring.

3. What we value and prioritize in a hotel stay changes over time. How do we account for that?

4. Prices fluctuate between peak and non-peak periods. How we do quantify this?

5. Our purpose of visit can be different, eg. going for a business trip vs honeymoon. The same hotel can be rated very differently because of this.

6. How do also we qualify, not just quantify, our overall experience?

After a somewhat deep reflection on these problems, we decided to Keep It Simple. Our “rating system” should work for us and help us answer the following questions:

1. The next time we visit this location, where had we stayed/ate/visited that is worth repeating?

2. When asked by a friend for recommendations, can we confidently give our recommendations of where and where not to stay/eat/visit? You, the reader, are a friend, and we want to confidently recommend stuff to you too!

#1 Rating Accommodations ❄

#2 Rating Food & Beverage Places 🍉

#3 Rating Attractions and Sights 🌈

Before you jump on us and exclaim that our “rating system” sucks, you are probably right. This system may suck for you, but we think it will work for us. We are gonna adopt this, and when a better system comes up, we enhance it.

How can we devise a better system? By reading your comments! Do comment and share your thoughts with us!

Our Proposal to the Problem

For the fun of it, let’s say this problem is within our paygrade, how can we begin to improve existing rating systems for hotels? Here are 3 suggestions.

#1 Filter Reviews for People Similar to Us

We tend to trust reviews from people similar us. We could be of the same nationality (eg. we value the ratings from fellow countrymen more), have the same purpose of stay (eg. on a honeymoon), travelling with the same crowd (eg. with kids), or have the same quirk (eg. light sleeper). Asking a few simple questions to personalise their searches beyond the usual ones (ie. price, location, ratings, etc.) can be really useful.

The ability to focus on just the reviews from profiles of people similar to us or our travel objectives can help us make informed decisions, and encourage us to pay-it-forward by leaving a review too.

#2 Ask for Their Recommendation

It takes effort to write a review. Some are happy to do so because they benefitted from the reviews of others. Most don’t mind leaving a simple rating, which is what most sites ask for. I like how Booking.com invites reviewers to elaborate on their reviews/ratings if they want to.

By adding one additional question to a simple rating - “Which groups of people would you recommend the hotel stay to”, the algorithm can surface searches relevant to the traveller’s profile and purpose. For example, a search result can inform that “20 people who stayed here recommend this hotel for families with elderly also”.

#3 Ratings Should Decay Over Time

A rating given 5 years ago should not have the same weightage as a rating given today because things can change. A high-rated hotel can get complacent or a low-rated hotel can improve over time. Making this transparent gives owners the right incentives to improve, and reviewers share or update their ratings.

This can also allow for new rating systems to be implemented as we allow old scores to fade away with time. We probably should have different “decay rates” for different things (eg. 5 years for hotels, 3 years for F&B).

How do you use current rating systems to help you plan better? Share your tricks and tips with us while we wait for rating systems to improve (which may never be). We look forward to reading your comments!