sondehub

patreon

sondehub posts

Posts: 42 Files: 14

webhorus and webwenet

A quick little update of something we've added for amateur sondehub users - web based decoders for horus and wenet.webhoruscan demodulate horus binary 4fsk and upload to sondehub using either input from a sound card (most browsers) connected to an SSB radio or from an RTL-SDR (Chrome only). Mark's made a demo videoof this working in action off a mobile phone!webwenetcan demodulate and upload to ssdv and sondehub using RTL-SDR (Chrome only). Like it also works (barely) on a mobile phone!Both can be installed as Progressive Web Apps, so t...

Graw sponsorship 2025, SondeHub Tracker updates and KA9Q support in radiosonde_auto_rx

Hi all!Apologies for the long time between updates here - we really appreciate your continuing support!Once again we are very happy to announce that GRAW Radiosondesare continuing their support of SondeHub. Their generous support allows us to keep SondeHub running and available to everyone. Without their support it would be much harder to provide the services and features we do.You can find out more about GRAW Radiosondes on theirwebsite.SondeHub Tracker UpdatesLots of work has gone into improving the performance of the SondeHub Tracker, and re...

DFM-17/DFM-09 mis-identification

Hi all,We are starting to see misidentifications of DFM-17 sondes as DFM-09 sondes becoming more prevalent over the US.We're working through a few options for how to better handle mis-identification of DFM17 sondes (based on serial number amongst a few other things), but this will take time to implement, and will probably need software updates at a majority of stations to work.For now, if you see a 'DFM09' with a serial number starting with 23 or 24 then it's most likely actually a DFM17 (very likely for 24).73Mark VK5QIFor upda...

March Updates

DiscordAfter having development discussions spread across multiple Discord servers for many years, we’ve decided to finally setup our own Discord server in an attempt to centralise things. We have channels for software development, and also general radiosonde hunting and amateur balloon launching discussion!If you’ve previously linked your Patreon and Discord accounts ( https://www.patreon.com/posts/how-to-link-your-96158411 ) then you will have already been added to our server. If not, you can join with this invite link: https://discord.gg...

GRAW Radiosondes continues SondeHub sponsorship for 2024

I am very happy to announce that GRAW Radiosondes are continuing their support of SondeHub for 2024. We are very appreciative of their support in 2023 which has enabled many new features to be developed on SondeHub. Their generous support allows us to keep SondeHub free to use at the service level our users have come to expect.We are looking forward to working and collaborating with GRAW Radiosondes again over the next year as there are some exciting opportunities for us to help each other.You can find out more about GRAW Radiosondes on their w...

Predictions - so many predictions

We recently made some changes to how our radiosonde predictions (for SondeHub not SondeHub Amateur) are delivered to clients.Previously the web client would request predictions for every single radiosonde. On the backend predictions are run on a schedule, so from a server load perspective having the client request all the data didn't create much additional load. However as the number of active radiosondes has grown over time, this has resulted in a larger and larger payload over time. Sometimes reaching 1MB. This is requested from the clien...

New experimental APRS gateway features

APRS receiver positionsOur APRS importer should now import the positions of stations that receive balloon telemetry.APRS chase car positionsAdditionally appending "SHUB" or "SHUB1-1" to APRS paths will result in positions being uploaded to SondeHub as chase car positions.

Frustum Culling [testing help wanted]

In computer game development there is a fairly common optimisation called frustum culling.In the example above, the players camera is on the right. When the player moves their camera around the game engine stops rendering all the objects outside of the view of the camera. Since the player can't see behind them, there's no point rendering all those extra bits of the game.So what does this have to do with SondeHub? Well SondeHub tracker is a fork of the HabHub tracker. HabHub was all about tracking amateur balloons. At anyone time there m...

Why and how we block old software versions

SummaryIf you are using rdz_ttgo_sonde software - upgrade to 20230427 / 0.9.3 or beyond ASAP. Failure to do so may prevent uploads to SondeHub.Why and how we block old software versionsBlocking uploads from old versions is not something we like doing. We try to avoid it where possible. However in certain cases bugs in decoders and software result in incorrect data being uploaded to SondeHub. This can take the form of: Incorrect positions Incorrect PTU (pressure temperature humidity) data Incorrect serial number generation Incorrect type identif...

New MQTT feeds for optimising the web interface

Back when SondeHub first got off the ground there were only a few tracking stations. We published all data to the web clients as it came in. Today there are 1300+ receiving stations feeding data and many older computers and mobile phones struggle with showing the unfiltered map.A longer term goal as we scaled up has been to improve the web experience by only receiving and drawing the data that's relevant to the user. Most people only look at a small section of the map and don't require the full feed.We already have topics in our MQTT th...

ARDC, Amateur tracker time periods, Amateur APRS-IS Gateway updates

Thanks for Mark (VK5QI) for writing todays patreon post!ARDC GrantBack in November 2022 we submitted a grant request to Amateur Radio Digital Communications, for funding for approximately 2 years of hosting costs. We are very happy to announce that our grant request was approved!https://www.ardc.net/apply/grants/2023-grants/grant-balloon-tracking-platform/Even with this grant, Patreon pledges are still vital to ensuring that SondeHub has funding for ongoing operation, and we are ever thankful to all of you that continue to show your support!Son...

What's been happening over Jan and Feb

Going to try to be brief (future me here - turns out this update was less brief than I thought it would be - we've been doing a lot!) because everything has been a lot lately but I wanted to cover some of the things we've been working on behind the scenes.MQTT WebSocket compression issuesFor several months we've been running "per-message-deflate" on our WebSockets solution which powers our tracker page and third party access. This has saved us significantly in bandwidth usage. However we've been seeing our WebSocket serv...

Map tile analytics

Hi all, We are currently looking into options for alternative map tile servers, however in order to do this we need to gain an understanding of how many tiles we are currently loading each month. To obtain this data we'll be temporarily capturing some very limited analytics. Data we will be capturing: - Number of tiles loaded for each map layer + random ID for each browser sessionData we will NOT be capturing: - Which specific tiles or locations are being loaded - IP addresses, callsigns, names, cookies Once we have a clear understand of m...

GRAW Radiosondes sponsors SondeHub

I am very happy to announce that we have a new sponsor and partnership for 2023, GRAW Radiosondes. You may have even had the chance to recover one of their DFM series of radiosondes. Their generous support allows us to keep SondeHub free to use at the service level our users have come to expect.We are looking forward to working and collaborating with GRAW Radiosondes over the next year as there are some exciting opportunities for us to help each other.You can find out more about GRAW Radiosondes on their website https://www.graw.de/

December updates

Logging reduction- Remove MQTT logs- Only log exceptions for ingestion- Moved monitoring dashboard to GrafanaOne thing that's always bugged me is how much we spend on logging useless info and the cost to have our pretty status monitoring dashboard. Fixing the first is a tricky problem as by default Lambda functions log their start and stop execution details - and the only way to turn this off is to turn off all logging.To work around this we have to create our own custom logger that handles just exceptions. That's now been implemented. ...

October updates

Listener websocketsA combined effort between Mark and I has moved listener (station) updates to via websockets now! This is a huge improvement in terms of cost and performance. When a chase car updates its position it should be reflected in the web tracker within a couple of seconds.Request countsWhile we can handle a large number of requests we still get charged per request. It doesn't seem malicious and is likely fixed in our latest tracker releases, it seems two users have been making millions of requests a day. We've implemented a W...

What's been happening?

It's been a little while since I've last posted, mostly because I've been busy, but that doesn't mean progress has stopped on SondeHub. Most of the teams effort has been in supporting the retirement of Habitat and Habhub.This has spawned a bunch of little jobs and projects such as : Documenting how SondeHub works for Habhub users Migrating Burst Calculator Fixing various SondeHub Amateur bugs Responding to API questions and queries Coming up with migration plansWe've also been planning performance improvements in SondeHub Am...

Grafana (trial/beta/something)

Before I kick off, I just wanted to mention that updates on the Patreon are probably going to be slow over the next months as I'll be travelling, but thats not to say progress on SondeHub will slow down. Mark and Luke are always improving the platform.Now onto Grafana. We've had a public OpenSearch cluster for sometime now, and while it does allow users access to query the data and make visualisations, accessibility has always been an issue. It requires logging in and finding the data that you are looking for. Due to AWS limitations the...

ARDC + AREG support

Eagle eyed users of SondeHub may of noticed some new logos on our loading screen from ARDC, AREG and of course Patreon. ARDC and AREG have teamed up to provide financial support for SondeHub. Between ARDC+AREG and our Patreon supporters we'll be able to ensure SondeHub remains operational for the at least the next year - and hopefully more!We are very grateful for their support and this funding allows us to focus on adding new features to the platform (eg, we likely wouldn't have been able to start working on the Amateur tracker without...

Amateur balloon tracker

For awhile I've mentioned that we've added endpoints in our API for tracking of amateur radio balloon launches, along with an APRS ingestion pipeline. Well we now have a tracker up!The tracker is available at https://amateur.sondehub.org/ and provides many of the features of the typical SondeHub tracker.Many thanks to Luke and Mark for the hard work getting this up and running.Please note that this is very much in beta, and we are still adding, improving and testing features. Things may break from time to time. Our next focus is improvi...

Database outage

We recently had a DB outage and I wanted to explain what went wrong.First some background. On Friday I upgrade our Elastic Search 7.9 cluster to OpenSearch 1.2 - this went well (however we are still monitoring performance of this change). This change was rolled out by myself manually. This was done to ensure a smooth deployment of the DB.Today during a deployment of some new amateur features to SondeHub (there will be some future posts about these) a terraform apply was executed. Terraform is the tool we use to manage our infrastructure configu...

An update about prediction failures

We were alerted that predictions were old / delayed. Upon investigating it appears that over time the function that processes our predictions was taking longer and longer to process predictions. It appears that in some edge cases some radiosondes were causing Tawhiri (the software we use for predictions) was taking over 30 seconds to process.When we had several of these events occur at once it pushed the latency of that function longer than its time out of 5 minutes and predictions would never finish. The cause of this still isn't fully und...

Amateur balloon tracking dashboards

We've been progressing a little bit on amateur balloon tracking and recently AREG launched a balloon "UPANDUP" which was tracked by several stations running the latest version of horus-gui which now contains support for SondeHub as an upload location.As an experiment I decided to play around with the ElasticSearch dashboards that we could use for analysing these sorts of flights.It was extremely fun to play around with the data and I can see this being useful for amateur radio balloon launches in the future. We may we port this dash...

Data transfer cost improvements

As SondeHub grows in the number of stations and the number of website users our data transfer costs increase. When there was only a handful for stations using SondeHub this wasn't a big deal but now that SondeHub but now that SondeHub is nearing 800 stations the data transfer fees are starting to add up. In the last month alone we've picked up an extra 100 stations! Added to this when we migrated from habhub it was done in a way to minimize the time to switch over. APIs were rushed into production and not much thought went into to optim...

XDATA and Radiosondy.info Recovery

XDATA Instrument DecodingThe radiosondes that we detect will sometimes include secondary instruments such as Ozone sensors which transmit their information via the XDATA protocol in telemetry. We can receive the raw transmitted XDATA values for RS41 and RS92 sondes with radiosonde_auto_rx with iMet-1/4 also coming soon. The DFM and M10/20 sondes also support XDATA but we cannot currently decode these values from telemetry due to a lack of sample recordings (if you are within range of a location launching either of these sondes with XDATA instru...

Skew-T and Historic data

The SondeHub tracker has been updated this month gaining two new major features, viewing historical launch site data, and generating Skew-T plots. The historical launch site data allows you to plot the landing positions for all sondes assigned to a specific launch site. The launch site modal has been expanded to allow for a year and month selection of data to display.The historical data is than loading from the AWS S3 bucket and we check if there is any recovery information for the sondes before we plot the downloaded data on the map. The last ...

Predictor performance improvements and ElasticSearch/Opensearch

Predictor UpdatesWe've been having issues with our home run predictor. When a prediction would run the server would lock and several requests would take a long time too come back. Previously we ran the predictor using a wind model that was shared via EFS (basically AWS managed NFS host). The predictor itself would memory map that file and read through it to provide a prediction.I suspect the issue is with the way NFS and the Linux kernel locks the file during memory mapped reads. To solve this I moved the files onto the ECS task (docker con...

Infrastructure as code

I've had this week off from work so I've had a little bit of time to work on various parts of SondeHub. Most of the work I'm doing has been around cleaning up our terraform configuration. Terraform is the tool we use to manage everything that's deployed in AWS.Our infrastructure has lots of moving parts, gateways, functions, CDNs, servers and everything in between. Each little aspect of our environment has many many many configuration options, like Python version, function name, memory usage, API path ect...While I can happily c...

Reverse Predictions

Regular users of the SondeHub tracker will be familiar with the live flight-path predictions run for all radiosondes showing up on the map. These predictions are run using the 'Tawhiri' prediction software (developed by Cambridge University Space-Flight - CUSF), which itself uses the NOAA's Global Forecast System weather model. We run our own instance of Tawhiri to avoid overloading the CUSF instance, pulling wind model data from the AWS mirror of GFS.Along with the live predictions, we added the ability for users to run flight-path...

Predictions update

A feature we've been meaning to get to is showing predictions for longer. What do I mean by this?Previously predictions only used the last 5 minutes worth of data to generate a prediction, however this was based on current time. So if the radiosonde had not been received in 5 minutes then the prediction wouldn't run. Some of our users enjoy trying to find radiosondes well after they've landed and after the radio transmitter had stopped so its necessary to have a good idea of where the radiosonde was likely to land.As of this afterno...