The UK’s Met Office does more than forecast weather on Earth—it does the same for space. This is because space weather events can have significant consequences for GPS, power networks, and other critical infrastructure. The office needs to work closely with forecasting centres in the US and Australia, and with academic partners. Migrating its mix of on-premises and hosted servers to Amazon Web Services (AWS) made it easier for these partners to access Met Office test environments and data. The migration was made possible with the help of AWS Partner. The service also provides high levels of resilience—critical for a system that must operate in times of crisis.

 

 

Opportunity

Forecasting space weather is a key responsibility for the Met Office, and space weather considered a significant risk on the UK’s National Risk Register. The Met Office provides forecasts and warnings for government agencies, emergency responders, and operators of critical infrastructure. Large space weather events will have a global, not just continental, impact, which can make it more difficult to find resources to restore systems.

The Met Office Space Weather Operations Centre (MOSWOC) works 24 hours a day, 7 days a week. MOSWOC works with centres in the US and Australia and needs the highest levels of availability.

Systems must ingest data from space and earth sensors and analyse it quickly to provide timely forecasts. To do this, the Met Office previously relied on a complex mixture of on-premises and hosted services. However, maintaining this infrastructure took up engineers’ time, leading to slow and manual releases for new features or forecasting models.

Solution

BJSS and the Met Office already had a successful working relationship after a 2022 user experience project. For the migration to AWS, the two companies worked together to build a system that met the specific requirements of a globally collaborative research and forecasting organization. Improved resilience provided by the network of AWS data centres was vital to the new solution. Forecasters work 24 hours a day, 7 days a week, and must always be able to rely on system availability.

Space weather is a potential disruptor to power and communications networks. And big events are exactly when space weather forecasting would be most important to inform and guide government and utility emergency response. “We need the most resilient systems possible,” says Mark Gibbs, Head of Space Weather and Transport at the Met Office. “We trusted AWS to provide that resilience while also providing good value for the taxpayer.”

Forecasters run their models using Amazon Elastic Container Service (Amazon ECS), which allows the Met Office to deploy highly secure, reliable, and scalable containers on AWS Fargate, meaning users can run containers without thinking about servers or clusters.

The Met Office is using Amazon CloudWatch to observe and monitor resources and applications on AWS. Amazon CloudWatch responds to performance changes, optimizes resource use, and gives the Met Office insights into the system’s operational health.

BJSS also configured Amazon CloudWatch to sends alarms to alert the right person to issues as soon as they arise. It also enabled composite alarms, a rule expression that accounts for the alarm states of other alarms to reduce alarm noise. This was needed to strengthen the Met Office security stance while reducing staff time spent on security management and dealing with false alarms.

Finally, the migration to AWS enabled BJSS to set up a continuous delivery/continuous deployment (CI/CD) approach to transform and modernize the Met Office’s development process and help to automate the delivery of code from test to production environments.

Outcome

The Met Office can now collaborate more effectively with the two other space forecasting centres in the US and Australia, and also with universities and researchers. “A key advantage of AWS is how it improves our ability to work with other partners,” says Gibbs. “Previously, providing access to on-premises systems was a real challenge. Using AWS, we’ve built a portal for academics to access our systems securely and run experimental algorithms. We can quickly adapt these for operational use.”

The new environment has also benefited the Met Office’s Ovation Service - a model that predicts where auroras will be visible on Earth in either the Northern (aurora borealis) or Southern (aurora australis) hemispheres. Video generation is required each time the forecasting model is run. This previously took Met Office systems 1.5 hours to complete for every forecast. Now this completes in three minutes with minimal infrastructure management.

With the CI/CD pipeline in place, developers can now make changes and get integration test results from an automatically deployed test environment in a matter of minutes. That process previously took several hours and many manual steps. A new release now takes just three clicks and minutes to run using an automated process. The new setup on AWS also provides backward consistency if a rollback is required.

Working with BJSS has also enabled the Met Office to move to a self-serve data model. That means both meteorologists and developers can get the data they need to run experiments more quickly and easily. Improving the data infrastructure also means that the Met Office is better able to take advantage of emerging technologies such as AI.

Learn more about our cloud capabilities here.