JMP

XMPPTwitterReddit
Featured Image

Newsletter: Voicemail Changes, Opt-in Jabber ID Discoverability

singpolyma@singpolyma.net

Hi everyone!

Welcome to the latest edition of your pseudo-monthly JMP update!

In case it’s been a while since you checked out JMP, here’s a refresher: JMP lets you send and receive text and picture messages (and calls) through a real phone number right from your computer, tablet, phone, or anything else that has a Jabber client.  Among other things, JMP has these features: Your phone number on every device; Multiple phone numbers, one app; Free as in Freedom; Share one number with multiple people.

This month sees the release of Cheogram Android 2.10.10-2, based on a new upstream version and with many bugfixes and small improvements, especially around the Command UI. We also now have our own F-Droid repositories for quick update of official builds from us. We have a repository for releases and for those who want to help testing new features as they are developed we also have a repository for pre-releases.

Some JMP customers forward their calls to another voicemail service, or otherwise do not have need for the JMP voicemail.  This month we added an official option to the Configure Calls command that allows disabling voicemail completely for users who need this.

The default voicemail outgoing message has been changed from saying “a user of JMP.chat” to specifying what JMP number has been reached.  Anyone with a name or nickname or custom voicemail greeting set is not affected by this change.

As a small improvement for multi-account billing users, renewal transactions now specify what number is being renewed by the transaction.

Cheogram (and thus JMP) is now allowing all users to opt-in to Jabber ID discoverability.  This is to allow users to discover the true Jabber ID behind a phone number so they can upgrade to end-to-end encryption, video calling, high quality media sharing, etc.  This is opt-in only, and most features that make use of this are not built yet, but we wanted to give people the option to express their consent now.  This is done as part of the registration process.  For existing users, if you do not want to opt in, there is nothing you need to do.  If you wish to opt in, simply run the Register command, choose JMP, and it will ask for your consent (it will show if you use the bot as Current Value true for technical reasons, but do not worry it is set to false unless you explicitly answer yes to that question.)

This month we have also made some progress with the early test phase launch of our data-only SIM and eSIM program.  The program is slowly rolling out to the waiting list over the course of the next month, as we gather data and feedback from early users.  If you are interested, adding your Jabber ID to the waiting list is still the best way.  We have also heard the interest in having these available for people who are not otherwise JMP customers, and hope to have that ready for testing soon as well.

To learn what’s happening with JMP between newsletters, here are some ways you can find out:

Thanks for reading and have a wonderful rest of your week!

Featured Image

Privacy and Threat Modelling

singpolyma@singpolyma.net

One often hears people ask if a product or service is “good for privacy” or if some practice they intend to incorporate is “good enough” for their privacy needs.  The problem with most such questions is that they often lack the necessary context, called a threat model, in order to even begin to understand how to answer them.  Understanding your own threat model (and making any implicit model you carry more explicit to yourself) is one of the most important steps you can take to improve your privacy.

What is a Threat Model?

A threat model is a list of possible vulnerabilities, often with attached priorities.  In the context of personal privacy, this includes anyone who you might not want to learn private information about you, what private information you most want that party to remain ignorant of, and why.  For example, someone may not want their ISP to learn that they are communicating on LGBTQ+ forums, because their ISP is their school and their school might tell their parents, whom they are not yet ready to tell.  In this example they might say “I don’t want the school to learn” but because of the reasons it may actually be more important to say “I don’t want my parents to learn.”  So the ISP, the school, and the parents all represent potential vulnerabilities, with the parents as the most important.

Why is a Threat Model Important?

You cannot protect your privacy unless you know what your are protecting and what you are defending against.  Otherwise you may take extra steps to secure something not worth protecting, omit something you were unaware needed protected, or even protect something at the detriment of something you would have cared more about.  Privacy is not a slider from zero to infinity, you cannot be simply “more” or “less” private in some general abstract way.

For example, someone may be a part of a group of insurgents in a small country.  They wish the contents of their communication to be kept a secret from the current government if any one of them is found out, so they choose to use an end-to-end encrypted messaging app.  They have prevented their mobile carrier and government from logging their messages!  They also secure their devices with biometrics so they cannot be stolen.  However, due to the unpopularity of this app in their country, when asked the carrier can immediately identify the current location of anyone using it.  When any of these people are brought in for questioning, the investigator forces the biometric (face or fingerprint) onto the device from the person in custody, unlocks it, gets access to all the decrypted messages, and let’s just say the insurgency is over.

So did the insurgents make “un-private” choices?  No!  For other people with different vulnerabilities, their choices may have been ideal.  But when their identity and current location is more at risk than the content of their messages, sending messages less-encrypted over a more-popular app or protocol (which could have all contents logged for all users, but very likely does not), and deleting them regularly from the local device in case they are caught, would have been more effective.

Privacy LARPing

“Privacy LARPing” is what happens when someone wants to be “more private” because it is cool and not because they have any well-reasoned need for privacy.  Believe it or not, this kind of use case also has a threat model.  The model may be more built on what kinds of vulnerabilities are currently trendy to defend against, but it exists nonetheless.  Putting thought and explicit description into your threat model can be a great way to seem even more “with it” so it’s highly recommended.  You may even identify real threats of concern (there certainly are some for everyone) and move beyond the LARP and into addressing your real needs.

How to Build a Threat Model

This is really an introspection activity.  Ask yourself what kind of entities are most concerning to you.  Estranged friends or lovers?  The other people at the airport or coffee shop?  Local police?  Local SUV owners?  Federal agencies?  Data brokers?  The list of people who may want to know more about you than you want them to is endless, so revisit your model from time to time.  Try to add to it and refine it.  This kind of work is never “done” because the scope is so vast.  Do talk to others and educate yourself about what the set of possible threats is, but do not take each new threat you learn about with the same weight.  Try to understand whether mitigations or new techniques are able to acheieve what you need, rather than blindly applying every “defense” without regard for context.

Signup with Cheogram Android

root@nicolosus.chat

Welcome to JMP.chat! If you are looking for a simple guide on how to sign up for JMP, then you have come to the right place! We will be keeping this guide up-to-date if there is ever a change in how to sign up.

We will first start with signing up from within your Jabber chat application on mobile, where you will never need to leave the client to get set up. I will be using the freedomware Android client Cheogram to do this signup. To start us off, we will need to create a Jabber ID (or “JID”). Upon first opening the app you will be presented with a welcome screen where you can choose to signup using the built-in flow for conversations.im or chatterboxtown.us, or you can choose your own custom server.

Main Startup Screen Jabber Server Selection Account User Creation

We will choose chatterboxtown.us for the purposes of this guide, but you are definitely free to choose whatever service you like, or bring your own! On the first screen of the server signup it will ask you to enter a username; this can be anything you want as long as it isn’t already in use on the server. After tapping Next, it will ask you to create a password for this account; length does not seem to be limited so create one as long as you want. Do not forget it, or use a password manager to create/store the password! Tapping Next again will log you in and offer to set an avatar for your account, you can set one now or choose to do so at a later time, if at all. Once logged in to the new account, Cheogram will ask for permissions to your contacts, you can accept or deny them. Accepting will allow us to implement the integration, which we will explain at the end of this post.

Password Creation Profile Avatar Main Chat Captcha Contacts Permissions Dialog

Now the fun begins! First we will need to add the Cheogram bot as a contact before communicating with it. To do this tap the “chat” bubble icon in the lower right corner, it will change views as well as change to a “+” sign, tap it again, then tap “Add contact”. A dialog box will appear where you can type in a contact’s “Jabber ID”, here you will want to put cheogram.com and tap Add, it will provide a prompt saying that it “appears to be a domain”, so tap “add anyway” and then it will open the screen for the bot, most likely displaying the bot comands tab view.

Chat Icon Press Add Contact Dialog Bot Commands Tab View

From here you will get to see one of the features of the Cheogram app, the Command UI! When a chat first opens you will notice that a “tab” near the top will appear titled “Commands”; it is from within this tab that you can manage all the features and settings of your JMP account, but first we need to register an account, and you will notice there is only one command available right now, “register with backend”. You will want to tap this “register” command, and it will take you to the next step. Here you will choose the backend SGX you are wanting to use; this list includes the most popular community ones as well, but we need to select the JMP option. The next screen will ask you to enter a search term for selecting a number; this can be by area code, state/province, city&state/province, or lead with the tilde “~” character to indicate a vanity pattern. We have chosen areacode 902 which then shows us a list of numbers on the next screen; choose one and tap Next.

Backend Selection Number Search Box Number Selection List

On the following screen it will ask you how you wish to activate your account; you have four options: Credit Card, Invite Code, Bitcoin, and Mail or Interac e-Transfer, as well as what the base currency is that you want to use. For this tutorial we have chosen credit card and Canadian dollars. Tapping Next will give us a page rendered in-app where you can add a credit card to use for activation, and choose the amount of funds that will be auto-charged to your credit card when your balance drops below $5. If you clear this box before tapping save, it will disable the auto top-up feature on your account, tapping Next will finish the sign up process. You can now open the Commands tab again and check out the new commands available for your account; details on these commands will come in a seperate blog post.

Currency and Payment Option Invite Code Activation Screen

Now that you have activated your account you are able to call (who really does this anymore, seriously? ;) ), SMS or MMS with your contacts. To add a contact within Cheogram is quite easy with the contacts integration. Now that the bot has been added to your account contacts, your device’s contacts should already be visible when you tap on the “chat” icon to start a new chat, or you can do the following to add a contact to your Jabber server. First tap the chat icon again like you did earlier to add the bot, tap the “+”, then “add contact”. The first thing you should notice that is different this time with the dialog box that pops up is that it now has two selectable buttons: Jabber ID, and PSTN. The PSTN option makes adding telephone numbers for calling or sending SMS to very easy, just type out the phone number you wish to add to your contacts and then tap ADD. This will automatically format the phone number according to the locale detected on your device. If you need to add an international number, you will need to add the phone in the full international format to override the country code being automatically added. With the contact now added, you can either start typing out a message, or tap the “phone” icon that appears at the top to make an audio call to the contact. Images, videos and audio files can also be sent using a number from JMP.

Bot Commands PSTN Contact Dialog Conversation View

Another helpful feature of the Cheogram app is native dialer integration. This allows you to make phone calls straight from your dialer just like any other phone call. To enable this, tap on the 3-dot menu in the top right of the main Cheogram screen, and then tap on Manage Accounts. Here you should see a new option under your account(s) that says “Manage Phone Accounts”, tap on that title and it will take you to a system settings page where you can enable your Jabber ID to make and receive calls, tap the toggle next to the account you want to enable. Now go back to your Manage Accounts screen in Cheogram and then tap the “gear” icon and from this second system settings page, you can select which “calling account” is your default, or to require it to always ask what account you want to use. Do note that there is currently a known bug in Android where this setting will reset after every reboot of the device, there currenltly is no fix out and Google says it will be fixed in a later release version of Android.

Manage Accounts Screen System Dialing Accounts Default Dialling Accounts

Featured Image

Newsletter: New Employee, Command UI, JMP SIM Card, Multi-account Billing

singpolyma@singpolyma.net

Hi everyone!

Welcome to the latest edition of your pseudo-monthly JMP update!

In case it’s been a while since you checked out JMP, here’s a refresher: JMP lets you send and receive text and picture messages (and calls) through a real phone number right from your computer, tablet, phone, or anything else that has a Jabber client.  Among other things, JMP has these features: Your phone number on every device; Multiple phone numbers, one app; Free as in Freedom; Share one number with multiple people.

This month sees the addition of a new member to the team, you will see him in the chat as seki.  Seki joins us as a software developer and general team member, be sure to say hi!

Cheogram Android 2.10.9-1 has been released.  This release includes a major new feature: the Command UI.  The best place to see this feature is when talking to the bot at cheogram.com.  You will see a new tab labelled “Commands” that lets you interact with the bot using a nice UI instead of by sending specially-formatted chats.  This release also includes several fixes to URI display and copying, and is based on Conversations 2.10.9 upstream.  We have added a long-press menu on the list of all active conversations to perform quick actions (such as “pin to top”), added support for muting yourself from the dialler integration, changed the ringback sound to be more familiar to USA and Canada users, and various other small bugfixes.

JMP is actively working on providing cost-effective data-only eSIMs and SIM cards for users.  Pricing is not yet final, and there is some work to do before this is ready for the general public, but if you are interested please sign up for the wait list at the link above.  Our first launch will be with USA and Canada coverage, but other areas are possible in the future if there is interest.

This month we are also pleased to announce the launch of multi-account billing.  This feature allows customers to have one account be billed for all their JMP accounts, or those of their family.  To get started with this, please contact support and indicate the accounts that you want linked together.

To learn what’s happening with JMP between newsletters, here are some ways you can find out:

Thanks for reading and have a wonderful rest of your week!

Featured Image

Newsletter: Multilingual Transcriptions and Better Voicemail Greetings

singpolyma@singpolyma.net

Hi everyone!

Welcome to the latest edition of your pseudo-monthly JMP update!

In case it’s been a while since you checked out JMP, here’s a refresher: JMP lets you send and receive text and picture messages (and calls) through a real phone number right from your computer, tablet, phone, or anything else that has a Jabber client.  Among other things, JMP has these features: Your phone number on every device; Multiple phone numbers, one app; Free as in Freedom; Share one number with multiple people.

As foreshadowed last month, the new voicemail transcription engine is now live for all customers who have transcription enabled (which it is by default).  This should improve speed and accuracy, and bring support for many more languages to the system.  Let us know if you notice any issues with the new transcriptions.

From the beginning of the voicemail system we have supported a default text-to-speech greeting if a custom one is not set.  The name used in this greeting is sourced from the customer’s legacy vCard if they have one set up.  We now also support modern vCard4 and PEP Nickname specifications to get this data, which should result in it working for many more people with many more clients.  Check the voicemail FAQ for details.

Many new JMP customers are also new to Jabber in general, and so our signup process usually suggests one or more free-to-use volunteer-run Jabber services that one can sign up with to get a working Jabber ID.  These services are best-effort by volunteers, and this month one of the ones most popular with our customers experienced an extended outage.  The best protection you can have against any kind of outage at your Jabber service is to have your Jabber ID be attached to a DNS name you control.  With or without your own name, we also include the option for any JMP customer to get an instance hosted by Snikket at no extra charge.  Please contact support if you have any questions about this.

To learn what’s happening with JMP between newsletters, here are some ways you can find out:

Thanks for reading and have a wonderful rest of your week!

Featured Image

Newsletter: Command UI and Better Transcriptions Coming Soon

singpolyma@singpolyma.net

Hi everyone!

Welcome to the latest edition of your pseudo-monthly JMP update!

In case it’s been a while since you checked out JMP, here’s a refresher: JMP lets you send and receive text and picture messages (and calls) through a real phone number right from your computer, tablet, phone, or anything else that has a Jabber client.  Among other things, JMP has these features: Your phone number on every device; Multiple phone numbers, one app; Free as in Freedom; Share one number with multiple people.

This month the team has been hard at work on a new major feature for Cheogram Android: the command UI.  This feature will get rid of the need to configure your account with a clunky chat bot on mobile, replacing it with a fit-for-purpose native UI that can be viewed under the cheogram.com contact.  And because we are implementing it using only open standards, the UI will also work for other command-using entities out there.  The feature is not quite ready for first release, but if you want to come test a pre-release just drop by the chatroom (see below for how to get to the chatroom).

Almost since the beginning of JMP one of the favourite features has been our voicemail.  Reading your voicemails instead of having to “dial in” somewhere and listen to them is a real advantage for many users.  However, the transcription is far from perfect, sometimes being slow and completely missing support for any language other than English.  We are now testing an alternative engine with anyone who is interested, this new engine gets you the transcription faster and supports dozens of languages.  Come by the chatroom if you want to help test this out before we roll it out as a full replacement.

To learn what’s happening with JMP between newsletters, here are some ways you can find out:

Thanks for reading and have a wonderful rest of your week!

Featured Image

Newsletter: Togethr, SMS-only Ports, Snikket Hosting

singpolyma@singpolyma.net

Hi everyone!

Welcome to the latest edition of your pseudo-monthly JMP update!

In case it’s been a while since you checked out JMP, here’s a refresher: JMP lets you send and receive text and picture messages (and calls) through a real phone number right from your computer, tablet, phone, or anything else that has a Jabber client.  Among other things, JMP has these features: Your phone number on every device; Multiple phone numbers, one app; Free as in Freedom; Share one number with multiple people.

This month our team launched a new product to help people looking to take even more control of their digital life by hosting their own social media instance.  Read about Togethr, what it is today, and a glimpse of our future plans.

JMP now supports SMS-only ports.  Landlines and most numbers with VoIP providers (but not numbers with most mobile carriers) are eligible to have JMP provide SMS/MMS messaging services for the number, while voice and other services would remain with the current carrier.  This feature is in Alpha, contact support if you are interested.

This month also saw the release of Cheogram Android 2.10.6-1.  This version merges in the latest upstream release of Conversations, as well as fixes for the contacts integration and playback for some media types (most notably 3GPP videos).

Finally, our integration with Snikket hosting is coming along.  For all of this year JMP customers have been able to get into the Snikket hosting beta with the promise of never having to pay for the JMP-using Jabber IDs they host there.  Now, JMP customers no longer need to contact Snikket staff to be put into the regular beta queue.  Contact JMP support and we can set you up with a Snikket instance directly.  We will continue to work on this integration until someday it becomes a fully self-serve part of signup.

To learn what’s happening with JMP between newsletters, here are some ways you can find out:

Thanks for reading and have a wonderful rest of your week!

Featured Image

Togethr: Soprani.ca Social

singpolyma@singpolyma.net

Last week we launched a sister product from the same team that brings you JMP: Togethr.  Why are we launching a second product?  Why now?  What does this have to do with the mission of JMP in particular, or the Sopranica project in general?

Togethr is a managed hosting platform for small Fediverse instances.  It is powered by the ActivityPub protocol that powers Mastodon, PeerTube, and so many others.  While there are several social networking solutions that build on XMPP (just like JMP does), and indeed we use one for this blog, we chose to go with something else for Togethr.  Does that mean we don’t have hope for XMPP in the social space?  No, rather it is an admission that the largest network for people to interact with in this way exists on ActivityPub-compatible software, and people need a solution they can use today.

As it grows, Togethr gives us the “skin in the game” motivation to bridge these worlds.  We are not the only ones interested in bridging the XMPP and ActivityPub worlds together, in fact the Libervia project is currently working on a grant to produce a first version of a gateway, that should be generally usable later this year.  We hope to eventually roll out an update that makes every Togethr instance seamlessly be both ActivityPub and XMPP without anyone needing to change their address.

Why not wait until “everything is ready” to go live with XMPP and ActivityPub at the same time?  Well, people need a solution.  Many people fleeing silos or otherwise being attracted to federated social networking find that self-hosting is too complicated, or they just don’t have the time to dedicate to it.  Many of these people end up creating an account on a giant volunteer-run instance, joining yet another silo (albeit a nicely federated one) run by admins they don’t know with financial and mental pressures they cannot understand.

Togethr gives people looking to federate their digital social networking experience full control without requiring systems administration knowledge or time.  Our team not only keeps the instance running, but provides support for users who may not be familiar with the software or the fediverse in general and need help getting everything set up.  However, there is no lock-in and people can easily move to another host or self-hosting at any time.  For example, if someone got an instance example.party and created the user person they would have address person@example.party just like you would expect on any Fediverse instance.  However, since they control the domain they could move to a different host or self-host, point the domain at the new instance, copy over their data, and no one has to “follow me at my new address”, everything just keeps working.

While we believe that single-user instances are the pinnacle of federation, Togethr does not limit the way people want to use it.  People may have family or friends they want to share posts with, who might not be motivated to join the Fediverse but will accept a personal invitation.  So every Togethr instance allows the customer to invite whoever they would like to join them on the instance, in order to smooth the onboarding for friends and family.  We hope that this can provide an option for people looking to take control over more of their digital life.

Featured Image

Newsletter: New Staff, New Commands

singpolyma@singpolyma.net

Hi everyone!

Welcome to the latest edition of your pseudo-monthly JMP update!

In case it’s been a while since you checked out JMP, here’s a refresher: JMP lets you send and receive text and picture messages (and calls) through a real phone number right from your computer, tablet, phone, or anything else that has a Jabber client.  Among other things, JMP has these features: Your phone number on every device; Multiple phone numbers, one app; Free as in Freedom; Share one number with multiple people.

The JMP team is growing.  This month we added root, whom many of you will know from the chatroom.  root has been a valuable and helpful member of the community for quite some time, and we are pleased to add them to the team.  They will be primarily helping with support and documentation, but also with, let’s face it, everything else.

The account settings bot has a new command for listing recent financial transactions.  You can use this command to check on your auto top-ups, recent charges for phone calls, rewards for referrals, etc.  There is now also a command for changing your Jabber ID, so if you find yourself in a situation where you are changing for any reason you can do that yourself without waiting for support to do it manually.

This month also saw the release of Cheogram Android 2.10.5-2.  This version has numerous bug fixes for crashes and other edge cases and is based on the latest upstream code which includes a security fix, so be sure to update!  Support for TOR and extended connection settings has also been fixed, a new darker theme added, and UI tweaks to recognize that messages are often encrypted with TLS.

To learn what’s happening with JMP between newsletters, here are some ways you can find out:

Thanks for reading and have a wonderful rest of your week!

Featured Image

Computing International Call Rates with a Trie

singpolyma@singpolyma.net

A few months ago we launched International calling with JMP.  One of the big tasks leading up to this launch was computing the rate card: that is, how much calls to different destinations would cost per minute.  While there are many countries in the world, there are even more calling destinations.  Our main carrier partner for this feature lists no fewer than 59881 unique phone number prefixes in the rates they charge us.  This list is, quite frankly, incomprehensible.  One can use it to compute the cost of a call to a particular number, but it gives no confidence about the cost of calls in general.  Many items on this list are similar, and so I set out to create a better list.

My first attempt was a simple one-pass algorithm.  This would record each prefix with its price and then if a longer prefix with a different price were discovered it would add that as well.  This removes the most obvious effectively-duplicate data, but still left a very large list.  I added our markup and various rounding rules (since increments of whole cents are easier to understand in most cases anyway, for example) which did cut down a bit further, but it became clear that the one-pass was not going to be sufficient.  Consider:

  1. +00 at $0.01
  2. +0010 at $0.02
  3. +0011 at $0.02
  4. +0012 at $0.02
  5. +0013 at $0.02
  6. +0014 at $0.02
  7. +0015 at $0.02
  8. +0016 at $0.02
  9. +0017 at $0.02
  10. +0018 at $0.02
  11. +0019 at $0.02

There are many sets of prefixes that look like this in the data.  Of course the right answer here is that +001 is $0.02, which is much easier to understand than this list, but the algorithm cannot know that until it has seen all 10 overlapping prefixes.  Even worse:

  1. +00 at $0.01
  2. +0010 at $0.02
  3. +0011 at $0.02
  4. +0012 at $0.02
  5. +0013 at $0.02
  6. +0014 at $0.02
  7. +0015 at $0.03
  8. +0016 at $0.02
  9. +0017 at $0.02
  10. +0018 at $0.02
  11. +0019 at $0.02

From this input we would like:

  1. +00 at $0.01
  2. +001 at $0.02
  3. +0015 at $0.03

So just checking if the prefixes we have so far are a fully-overlapped set is not enough.  Well, no problem, it’s not that much data, perhaps I can implement a brute-force approach and be done with it.

Brute force is very slow.  On this data it completed, but as I found I kept wanting to tweak rounding rules and other parts of the overlap detection the speed became really problematic.  So I was searching for a non-bruteforce way that would be optimal across all prefixes and fast enough to re-run often in order to play with the effects of rounding rules.

Trie

As I was discussing the problem with a co-worker, trying to speed up lookups we were thinking about trees.  Maybe a tree where traversal to the next level was determined by the next digit in the prefix?  As we explored what this would look like, it became obvious that we were inventing a Trie.  So I grabbed a gem and started monkeypatching things.

Most Trie implementations are about answering yes/no questions and don’t store anything but the prefix in the tree.  I wanted to be able to “look down” from any node in the tree to see if the data was overlapping, and so storing rates right in the nodes seemed useful:

def add_with(chars, rate)
    if chars.empty? # leaf node for this prefix
        @rate = rate
        terminal!
    else
        add_to_children_tree_with(chars, rate)
    end
end

But sometimes we have a level that doesn’t have a rate, so we need to compute its rate from the majority-same rate of its children:

def rate
    # This level has a known rate already
    return @rate if @rate

    groups =
        children_tree.each_value.to_a         # Immediate children
        .select { |x| x.rate }                # That have a rate
        .combination(2)                       # Pairwise combinations
        .select { |(x, y)| x.rate == y.rate } # That are the same
        .group_by { |x| x.first.rate }        # Group by rate
    unless groups.empty?
        # Whichever rate has the most entries in the children is our rate
        @rate = groups.max_by { |(_, v)| v.length }.first
        return @rate
    end

    # No rate here or below
    nil
end

This algorithm is naturally recursive on the tree, so even if the immediate children don’t have a rate they will compute from their children, etc.  And finally a traversal to turn this all back into the flat list we want to store:

def each
    if rate
        # Find the rate of our parent in the tree,
        # possibly computed in part by asking us
        up = parent
        while up
            break if up.rate
            up = up.parent
        end

        # Add our prefix and rate to the list unless parent has it covered
        yield [to_s, rate] unless up&.rate == rate
    end

    # Add rates from children also
    children_tree.each_value do |child|
        child.each { |x| yield x }
    end
end

This (with rounding rules, etc) cut the list from our original of 59881 down to 4818.  You can browse the result.  It’s not as short as I was hoping for, but many destinations are manageable now, and thanks to a little bit of Computer Science we can tweak it in the future and just rerun this quick script.

Creative Commons Attribution ShareAlike