Whether it’s a Michelin-star chef or your favourite local baker, nobody likes giving away their recipe for success. Similarly, you rarely get a glimpse of what really goes on behind the scenes of big comparison tests. We want to change all that, so here’s your chance to come with us to Spain, take a look at our test track, and experience a taste of everyday life in the office during our mega group test of 30 eMTBs. All from the comfort of your own home, of course.
Picture this: a secluded finca in the middle of the Spanish mountains, a pizza oven on the balcony, breathtaking trails as far as the eye can see, and 30 of the hottest eMTBs in the garage, all chomping at the bit. Sounds fantastic, right? It is. But in a way, it’s also too good to be (totally) true. It might all look like fun and games from the outside, but a project of this size doesn’t produce and write itself while we’re out there shredding the trails 😉 There’s also our fair share of blood, sweat, tears and tires.
Our core mission is, of course, to help you buy the right bike and to provide the know-how and insights you need to do so, not to mention well-founded opinions about the latest technologies and developments to inspire you to make the most of the world’s best hobby. This article takes a slightly different approach, but hopefully it will inspire you: to jump into things, try something new, make some mistakes, and maybe grow as a person in the process. Primarily, we want to give you an idea of the huge project and the team that’s behind it. It’s our “recipe for success” for the biggest group test we’ve ever done.
The enormous scope of the project was obvious from the beginning: we wanted this to be the biggest, best overview of the market we could possibly provide – and therefore the biggest group test our publishing house has ever organised. The result? 4 months of toil for 10 editors with countless heated debates, aching legs, some nasty cuts and bruises, and more than 200 pages of text (A4, size 10 font) at the finish line. Per language, of course – we had to do everything twice for our German and English editions. See, we don’t just swan around riding cool bikes and drinking expensive coffee all day!
The good old days
Before going into the project on hand, let’s venture back to the early days of our magazines. Without the experience we’ve accumulated in the 10 years since the inception of E-MOUNTAINBIKE magazine way back in 2013, a group test on this scale would have been unthinkable. For one thing, we couldn’t have ordered, tested, photo shot, and written about 30 bikes simultaneously – never mind the fact that there weren’t even 30 eMTBs on the market at the time if we had wanted to. Also, we didn’t have the necessary team size and weren’t savvy enough between us to put so many e-MTBs into context and delve into the complexities of the bikes and their motor systems. But being the first eMTB magazine on the market, we were not only part of the development from day 1, we also helped shape it. So, now we reap the benefit of years of experience, and it’s this hard-earned experience that has allowed us to take on a mega group test of this scale in the first place.
Bikes, Bikes, Bikes
The idea and the concept are born. What’s needed now? Bikes! Well, “what’s the big deal?”, you might think. Pick out a few cool eMTBs, go ahead and order them – easy! Well, not as easy as it may sound. For one thing, we didn’t want to order just any old eMTBs for this group test. The contenders have to be interesting as well as relevant. So, first things first: Which are the most fascinating models out and about? Which are the most promising, innovative, and fun? And most importantly: will our readers see eye-to-eye with us? To find that out, we consult our reader survey, and (of course) our entire editorial team. Brands that you’re particularly interested in (typically the top 3 from our reader survey) and manufacturers with exciting concepts or promising technologies get the chance to enter multiple bikes in the test.
So far so good. This process yields around 80 eMTBs that get long-listed for the test – and we’re only talking about models in general, mind you – we’re nowhere near discussing spec variants. The vast offering of eMTBs available on the market nowadays boggles the mind and can be pretty overwhelming, both for consumers and also for us. It can turn the buying process into a nightmare. With that sort of range of products, it’s so easy to end up spending your hard-earned money on a bike that looks perfect, but just isn’t Mr (or Mrs) Right for you.
So, here we are after the first bike brainstorming sessions: list in hand, faces set, and we’re ready to fight it out in a series of editorial meetings. Who’s going to be invited? Some tough decisions have to be made, and market relevance, reader interest and novelty factor all have to be taken into account. It goes without saying that you can’t be everybody’s darling in a market that big, with interests so diverging. What’s more, there are some manufacturers, such as Canyon and Cube, that we would have liked to include in the test but who decided to opt out. There are many reasons for such a decision: from simple availability issues to marketing strategy, to (unwelcome) critique in past tests and the simple concern not to be able to hold one’s own in such a large field of competitors. By this point, brands have realised that we don’t mince our words. On the other hand, there are bike manufacturers who’d like to participate, but don’t have a suitable model in their portfolio, or have already contributed the same bike to a previous group test, so their bike wouldn’t add much value to our test field.
Well, no matter the result of the test: At the end of the day, the group test benefits every participant. Our E-MOUNTAINBIKE group test isn’t just a sort of “world championship” – it shows off some cool bikes to readers all over the world.
As soon as the dust has settled in the office, after some serious head-butting, we’re ready to send out the invitations. Participation in our group test is free of charge and the core factor is relevance. Quite regularly, therefore, manufacturers with whom our relations are less than cordial are still part of a test. We notify the manufacturers in our email that we reserve the right to publish the invitation, so we are giving you a little insight 😉
We’re looking for the best all-rounder with the most convincing overall concept which delivers fun and security on the widest range of trails and can cover everything from relaxed tours to flow trails and techy singletrails. The winner doesn’t necessarily have to be the best in all individual disciplines. Instead, it should offer the best overall package and combine the best all-round qualities. Simultaneously, it should be just as suited to beginners as it is to experienced riders. Intuitive and balanced handling, uphill, downhill and touring abilities, design, motor performance, battery concepts, weight and smart details are just as important as a component spec that suits the bike’s intended purpose. Our evaluation criteria also include user-friendliness (smart usability) and user experience (rider experience). However, the eMTB market is becoming increasingly diverse, making it impossible to cram all existing eMTBs into one category. Therefore, we’re looking for a specialist that stands out from the crowd of modern eMTBs, one that can impress in individual disciplines. We will test the bikes within their potential and the intended use specified by the manufacturer and assess them according to the corresponding range of applications. The best bikes in test will be awarded our coveted “Best Buy Tip”. We don’t measure value for money based on spreadsheets, theoretical data or the price tag of components. Instead, we look at real-world performance and the quality of the complete package.
After answering every question and calming any qualms, which often involves emails as long as your arm and endless phone calls, the bikes are finally on their way to our office. Once the bikes have arrived, we carefully measure and weigh them, and meticulously list all of the components. Then we compare our results to the available models of each manufacturer to make sure that the eMTBs available to the public have the exact same specifications. You can probably imagine how much time it takes to do that for 30 bikes.
The fun part
Our editorial office, as the discerning reader may already have sussed out, is located in Leonberg, near Stuttgart. In January, trail conditions in this region leave something to be desired, to say the least – not that we have to tell you that. This means finding a suitable test location meeting the following requirements: accessible by car, consistent testing conditions (read: little rain), and – please, please – no sub-zero temperatures. And of course: trails. Fast, gnarly, flowing trails with flair… And if it’s not asking too much, we would quite like to have a nice time off the trails too, and maybe provide some destination inspiration for your next bike trip. We finally decide on a solitary finca in the mountains of Spain. To be precise: in the region around Santa Coloma de Farners, roughly 30 minutes from Girona. This remote finca sounds like the perfect HQ for our test – if not for the extremely rough access road, as we’re soon to find out. We’ll just keep the name of the company we rented the Sprinter from a secret ;). And before you ask: Yes, we’re packing all 30 eMTBs plus all our stuff into one Sprinter – why make life easier than it has to be?
No sooner have we arrived than we set to work unloading the van and hooking the bike batteries up to the power, before finally sitting down and clinking glasses. What we admittedly have completely overlooked is the fact that the remote finca is powered solely by solar panels, which is to say: it isn’t connected to the power grid. Definitely connected, on the other hand, are 30 eMTBs… in January, no less. (Here’s a big thank you to the owner of the finca, who from day 1 filled up the emergency generator with gas out of jerry cans.) So, we work out a battery charging plan for our bikes, learn some electrical engineering on YouTube on the side, and down quite a quantity of the excellent house wine to drown out the incessant, monotonous drone of the generator. It is decided that we need a Charging Supervisor, the dubious honour of which goes to our trip mechanic Eric, who has to check several times a night whether all bikes are still charging dutifully. Our YouTube study has taught us that not every charger can handle power spikes – and the little blighters use any excuse to play up. Ha, but not on our watch, they’re not!
Now for some fun: The entire trail network has to be checked out and the test tracks are chosen. We need as many different trail characteristics as possible. Our two test trails have to include flowy descents and dusty berms, nasty rock gardens, tight hairpin bends, steep descents and challenging climbs – plus everything in-between. Luckily, the journey to the trailhead is already part of the game. Therefore, there is a little bit of everything: from flat gravel roads to impossibly technical climbs (for most of the bikes, at least). By the time all the trails are mapped out, we’ve already been here for two days. But the essential thing in a production like this is to know the area inside and out. You need to find the perfect test track, and know all the best spots to capture it all on camera.
Six full days of testing lie ahead of us. What that means is: riding the same trails all day, every day for 6 long days, so that each of the 10 testers can ride each bike in direct comparison. It also means that each tester has to swap the pedals and adjust the cockpit, saddle position, suspension and tire pressure each time, and make extensive notes and necessary adjustments after each ride – all that, well, times 30… On top of that, technical problems with hardware and software have to be solved, and defects repaired. Bent cranks, torn cables, broken or defective displays and mangled derailleurs are all part of the fun, on top of the usual flat tires. After sundown, the most important emails have to be answered, some unrelated parallel projects have to be coordinated, and one or two articles have to be published – our website isn’t on holiday and needs at least some new content, even with almost the entire team toiling away in Spain.
After six days on the test trail, each tester has climbed roughly 11,000 metres in altitude, and has around 220 km of trails in their legs. Sure, they’ve had some help from their eMTBs, but it still takes its toll. Luckily, we survived all the spills with mere scratches and bruises, and we wisely skipped the last lap each day 😉
DIY
Arguably the most important ingredient in our recipe for success is our team. Everything – ideas, organisation, testing, pictures, and copy writing, right up to the point of publication – is done by the same people. The know-how always gets handed down through the team. That’s why we don’t need external photographers, celeb testers, or professional models. The entire group test in question, for example, was photographed by Peter and Mike, both part of our editorial team, who were also involved in the organisation as well as the testing and texting. And each member of the team that isn’t found behind the camera is found in front of it: the riders that you see in the photos. That they all look so handsome is due to the fact that we choose our editors solely for their good looks ;). But seriously, that’s how we remain flexible, and are able to use our budget for long articles and huge projects such as this one. In addition to the testing, the production of the pictures also takes an immense amount of time. For example, we spent several days shooting action photos as well as detailed pics of the bikes, while checking out all of the additional features and software solutions on the side. Also, the large overview photos – those that show all of the test bikes together – are quite a piece of work. Just imagine trying to align 30 bikes perfectly… It takes time and patience. But quite apart from that, such a picture requires a lot of planning and logistics. More than a terabyte of pics was taken during this group test, by the way. Pictures that all have to be sorted and edited, one by one…
Editor or star chef?
Each member of our team has special strengths (and weaknesses) that have to be considered in such a production. There are, for instance, the hangry guys, who sit around being, well, hangry, but also some gourmands, who supply us with delicious food every morning and evening. Nothing beats a fancy muesli, the notorious avocado and poached egg on toast, regional specialities, yummy pinata, or homemade pizza.
The hardest week
All of our impressions and insights are jotted down by each tester during the testing phase. It’s important for everybody to shut up about their own findings and opinions so that a) there’s no quarrelling (yet) and b) everybody can make up their own mind about the bikes. Back at the office, on the other hand, things start to heat up. Arguably the hardest, most tedious, and yet most important part of any test is the phase where everything is laid on the table and discussed openly, with the goal of reaching a sound conclusion. This will take roughly 90 minutes per bike – times 30, as always. How should you imagine that process? Heads spin, sparks fly, and things can get a little heated, especially after we’ve spent an hour discussing the minutiae of removing a battery. Once the dust has settled and tempers have cooled, any remaining ambiguities are cleared up with calls to the manufacturers, and some additional test runs on our home trails.
Making it work
After all that buzz, the lengthy process of putting it all into words begins, resulting in no fewer than 200 A4 pages of size 10 text per language. Each text is checked by at least three other people to make sure that the information is correct, and presented in a clear, relevant manner that will benefit the reader – you. Also, the final versions of all 30 texts are compared to each other to check that the results are still correct after being transformed from notes to text. After that, our international team get to work, with translators and proofreaders all over the world from South Africa to England, Spain to Switzerland, and of course Germany. Not to forget that all the pictures have to be edited and uploaded. This is where our graphic designers come in, creating illustrations and formatting the images so that you can enjoy a convenient, free read in our app. At the same time, the publication on our website is prepared. All web pages – 62 to be exact – have to be built, interlinked, checked, and published. As soon as everything is online, our newsletter goes out, and posting on different social media channels starts. Also, the manufacturers are notified separately. Feedback begins flooding in, from readers and manufacturers alike. Most of them are happy with the hard work we have put in, but there are some interesting criticisms too. We’re known for our direct approach, and not every brand or reader appreciates that. By the time everyone is busy reading the results of our group test, we’ve already moved on to organising the next one. In this case, our Best eMTB for under € 7,000 group test went online only a few weeks later.
Take it or leave it tell us
As you can see, this is a lot of work for content you can read for free, both in our interactive app and from our website. All of this is made possible by a dynamic, motivated team that love doing what they’re doing and bring a huge load of enthusiasm to the table. Most of us have turned our backs on our former jobs because they weren’t the real thing. Working time is still lifetime, and there’s a lot of it. And if you can spend it shredding trails somewhere in Spain, so much the better 😉 Our team has a lot of career jumpers, but if the basics are sound, we are ready to train everybody with practical know-how and journalistic skills. The learning and development curves are enormous. And that’s the essence of our work and mindset: we love challenges, and learning about new stuff. Which is why we don’t leave it at just testing e-MTBs, but also gravel bikes, urban cargo bikes, and even electric cars. This broadens our horizons and provides a wealth of experience and perspectives, as what you learn in one segment benefits another.
A heck of a lot of person-power goes into these group tests. Instead of flooding you with daily news and press releases, we try to focus on high-quality content. So don’t be surprised if you don’t find a new article from us every day, but rather look forward to when there’s silence again, because that means we’re working on our next big project. Of course, we also look forward to your feedback, whether it is positive or critical. We promise to read every single message that comes our way, even if we might not be able to reply to each one.
If you have something to share with us right away, feel free to get in touch: hello@ebike-mtb.com
Did you enjoy this article? If so, we would be stoked if you decide to support us with a monthly contribution. By becoming a supporter of E-MOUNTAINBIKE, you will help secure a sustainable future for high-quality cycling journalism. Click here to learn more.
Words: Peter Walker Photos: Diverse
