“What’s New?” is a series of blog posts covering recent changes to Comet in more detail. This article covers the latest changes in Comet Voyager over June 2023. 

There were six Comet software releases during June – two in our quarterly 23.5.x Thebe release series, plus four releases in the 23.6.x Voyager release series.

Policy-linked Protected Items​

Comet has a comprehensive Policy system that allows you to apply rules to entire sets of customers at a time, such as restricting their ability to open the desktop app or restricting what features and settings can be used.

One of the existing Policy options allowed you to create a Protected Item that would be applied by default to all new devices that register in the account. However, it was limited to when the device was registered for the first time, it only defined the default settings, and after that the Protected Item could drift from what was configured in the policy. Having a default value that can then change is an important use case, but over time, we heard feedback that people would like these default Protected Items to remain tied to the policy version. In Comet 23.6.0, we made that possible.

You can choose whether the Policy Protected Item applies only for new devices, or if it remains linked to the policy after creation. This allows you to configure a Protected Item for a specific application, file-path, or integration target; apply it by default to all users of the Policy; and still be able to change those settings centrally for all users of that policy.

This was a highly popular request on our Feature Voting page. Thank you to all the partners who shared their support for this feature. If you haven’t seen this system before, the Feature Voting system allows you to upvote existing requests, raise new requests, discuss features with other Comet partners, and get notified when a change happens in the product. We look at this system often to help guide the priority of new development.

Extended admin permission options​

As an administrator, when configuring permissions for other administrator accounts, additional options are now available.

These new options allow you restrict the capabilities of other administrators. This is intended for the use case of tenant administrators, or limited-permission administrators of a shared Comet Server. The use cases are varied but are often from the perspective of restricting the visibility of the Comet branding for white-label purposes.

• You can hide the Server History and Server Info widgets from the homepage, including Comet’s version numbers and update information. This simplifies the main screen for the limited-permission administrators who would be unable to act upon this information;
• You can enforce that the target administrator is unable to create new Storage Vaults via Storage Templates or via custom storage; and
• You can filter and restrict which cloud storage providers are available for these limited-permission administrators to use.

This work was also inspired from our Feature Voting system. Thanks to everyone for your patience as we combined several separate requests to refine down the essential requirements for this use case.

These features are also available in the Comet Server API and in our latest SDK releases for PHP, Ruby, Javascript / Typescript, Go, and C# / .NET.

New minimum system requirements for macOS​

From Comet Backup 23.6.3 onwards, your customers must have macOS 10.13 High Sierra (2017) or later to continue running new versions of Comet Backup.

If your customers are still using macOS 10.11 El Capitan (2015) or 10.12 Sierra (2016), the last releases that can be run on these older versions of macOS are the Comet 23.5.x Thebe series, or 23.6.2 Voyager. These versions of macOS are no longer receiving security updates and we would recommend that all end-users upgrade if possible.

There is no change to Comet Server’s minimum OS requirements as we do not offer Comet Server on macOS. It is safe to use the bulk deployment system in Comet Server to continue deploying software updates to all macOS customers – the affected users will skip the install and remain on their currently installed version of Comet Backup.

We last upgraded the minimum macOS version back in September 2022. We expect to be able to keep the 10.13 baseline for a longer period of time.

“Run when PC starts” now includes Sleep​

Comet has long supported a “Run when PC starts” option in the job schedule settings. This option is highly useful for laptops that are often offline or sleeping, where a regularly scheduled backup job is unlikely to occur at the odd moments when the device is online. By running a backup job immediately when the PC boots, the device is more likely to complete a successful backup job.

However, recent versions of Windows have expanded how widely “Fast Startup” is applied across different PCs, to the point that a Windows laptop will often not restart, but only simulate a restart via hibernation-related techniques. This has a negative effect on the “Run when PC starts” option as it was only triggered via a full restart, which is not the default behavior in current versions of Windows.

As a result, we have extended the “Run when PC starts” option to count waking up from sleep as “starting the PC”. This is more in keeping with the behavior of current-generation laptop computers and it should make the option work as-expected in more cases. Compared to the old behavior, it is likely that additional backup jobs could be run unexpectedly, but our expectation is that this is not a bad thing, and the result will be more reliable overall.

Comet Server web interface enhancements​

The 23.5 Thebe release last month included a major redesign of the Comet Server web interface. We’ve continued to build upon this throughout June in the 23.6.x Voyager series with many enhancements for server operators.

One area of focus was on ease-of-use. When configuring a Protected Item for Application Aware Writer, Hyper-V, or Microsoft Exchange, you can now use the top-right button to remotely browse the customer’s PC to select items for backup. Compared to finding individual virtual machine IDs, this is a significant improvement to the ease-of-use of this feature.

Additionally, when saving changes in the Settings page, the page will now intelligently wait for settings to be applied, instead of leaving you at a web browser refresh screen. This makes for a much more intuitive and user-friendly experience. The Comet Server web interface will now also warn you more clearly about invalid bucket names for cloud storage.

There was also a particular focus on performance enhancements for the Comet Server web interface. Loading the web interface should be snappier and with fewer loading spinners, as the necessary information is able to be loaded in parallel, as well as reused from previous lookups. The Quick Search area is now faster and has a new design style. When remotely controlling a connected device to start a restore job, the restore job will reuse information known by the web interface instead of recalculating it, resulting in a restore job that starts faster and finishes faster.

A third area of focus throughout the month was on fixing known issues that have been reported. Thank you to all partners who reported issues – the latest 23.5.x and 23.6.x updates should have addressed all the web interface issues raised to this point.

CometCon 2023​

Following on from last year’s successful event, we hosted another week-long meetup with Comet team members from all over the country. With a strong focus on technical talks, networking, and a special coffee-tasting event, everyone had a great week! I’d like to extend a special thank you to our guest speakers!

That’s all for this month – the blog will return next month with more news about all the latest changes to Comet.

Changes compared to 23.5.0

NOTICE: The “Run when PC Starts” option will now also apply to devices waking up from Sleep

New Features​

• Protected Items defined by Policy can now optionally remain linked for future changes
• Added several new admin permissions to allow a global admin to help prevent a tenant admin from seeing the Comet service or storage provider types in use. The global admin can hide server history and server info widgets on the dashboard, prevent creation of storage via templates or custom storage, and can also filter the list of allowed cloud storage providers

Changes compared to 23.5.0

NOTICE: The “Run when PC Starts” option will now also apply to devices waking up from Sleep

New Features​

• Protected Items defined by Policy can now optionally remain linked for future changes
• Added several new admin permissions to allow a global admin to help prevent a tenant admin from seeing the Comet service or storage provider types in use. The global admin can hide server history and server info widgets on the dashboard, prevent creation of storage via templates or custom storage, and can also filter the list of allowed cloud storage providers

Thebe is the latest entry in our quarterly rollup series. It branches off from our main rolling Voyager development into a fixed target for our partners to qualify and build upon. 

Like Comet’s previous recent quarterly software releases, ‘Thebe’ is named after a moon of Jupiter, which in turn takes its name from an ancient Greek mythological figure. It is a very small moon with only 0.004x earth’s gravity, meaning it is less likely to shape itself into a sphere – in fact a large impact crater covers around 40% of its surface area.

It was discovered by the Voyager 1 and 2 spacecraft in 1979, but owing to its small size and position, almost nothing more could be discovered until the next spacecraft Galileo visited two decades later. Scientists believe it contains water ice.

As for the software, Comet 23.5.0 ‘Thebe’ brings 10 features and 22 enhancements, including S3 Object Lock support for immutable protection against ransomware, a new design for the Comet Server web interface, an official Docker container, a C# SDK, support for Azure Key Vault codesigning, and more.

As always for a new quarterly release, there are two changelogs for 23.5.0 ‘Thebe’ depending on whether you are coming from the previous quarterly release or the previous Voyager release:

Changes compared to 23.2.2

Breaking Changes​

This release makes some changes to how codesigning works in Comet Server, which includes bundling a PKCS#11 engine.

• If you are using a hardware dongle for Authenticode EV codesigning, you may require an updated codesigning configuration
• Performing Windows codesigning from an ARM64 Linux version of Comet Server now requires glibc

New Features​

• Cosmetic overhaul for the Comet Server web interface

• An official Comet Server docker container, which offers MSPs another deployment option. For more details, visit this link
• Support for Azure Key Vault as an option for Windows Authenticode codesigning

• Ability to use S3 object locking
• New option to permanently remove files marked for deletion in versioned S3 storage
• New Comet Server home page widgets for Protected Items types, server storage, server self-backup, and server replication
• When configuring an exclusion via Policy in the Comet Server web interface, default suggested exclusion paths have been added
• Ability to search for items to restore within Storage Vault snapshots via the Comet Server web interface
• A Test Connection button has been added to Storage Template configuration dialog
• Authenticode codesigning has been applied to the Comet Backup client uninstaller on Windows

“What’s New?” is a series of blog posts covering recent changes to Comet in more detail. This article covers the latest changes in Comet Voyager over May 2023.

There were four Comet software releases during May, all in the weekly 23.3.x Voyager release series. The work this month included a mix of core application development as well as projects in the surrounding ecosystem:

Learn about Chunking​

Earlier in May, Mason gave a presentation of the history and overview of backup technologies, starting with simple file copies, through chain-based approaches, and culminating in the Content-Defined Chunking technology used by Comet.

Comet Server in Docker​

The Self-Hosted Comet Server is now available as a Docker container:

Docker is a popular system for running applications in “containers”. In the same way that freight shipping containers have a consistent size and shape, in order to fit within a larger shipping ecosystem, Docker containers also have some consistent behaviors in order to fit within a larger application runtime ecosystem: a “containerized” application has a standard way to start, stop, and log messages; how their storage and configuration is managed, how ingress and TLS termination works, and how sandboxing and permission restrictions are applied.

By offering Comet Server as a Docker (OCI) container, it should be easy for existing Docker users to get started with the Self-Hosted app and to manage it going forward. It also enables new functionality such as running multiple self-hosted Comet Servers on the same PC or VM. This does not replace the existing Self-Hosted installers, which will continue to be available.

There has been a long-standing open feature request for an official Docker container. Thank you to everyone who voted and commented on this feature, and a special thanks to “Hobadee” for providing an unofficial solution to the community so far.

C# .NET SDK and expanded API documentation​

When you visit the the Comet Server web interface, your web browser loads and runs a “Single Page App” (SPA) locally inside your browser. This webapp speaks to the Comet Server backend solely using our public API. This API broadly covers every action you take within the Comet Server web interface, which means if you can do an action in the web interface, then it can be automated by a developer.

The Comet Server API is comprehensively documented, and during May we added many more comments and explainer notes to the constants, endpoints, and type definitions. Developers using the existing PHP SDK, Javascript / Typescript SDK, Ruby SDK, and Go (Golang) SDK will see these new comments appear in their IDEs during autocomplete.

During May, we put the finishing touches on the latest addition to this group – a new public .NET SDK. It’s available as CometBackup.CometAPI.SDK on NuGet and is developed in C# for use with any compliant .NET Standard 2.0 runtime, including .NET Framework 4.6, .NET Core 2.0, .NET 5, or any later versions.

Getting started with the SDK should be straightforward:

using (var client = new CometAPI("http://127.0.0.1:8060", "admin-user", "admin-password")) {
    var users = client.AdminListUsers();
    foreach (var user in users) {
        Console.WriteLine(user);
    }
}

Every API method is also available as an Async() version to allow for concurrency.

Our SDK version numbers generally adhere to Semantic Versioning and any enthusiastic readers may have noticed this SDK uses a “version zero” version number at present. We don’t necessarily expect to make breaking changes, but we are actively requesting feedback about this new SDK from existing C# users to ensure we’ve created something idiomatic. We’d love to hear any feedback about the SDK design or any bug reports before we commit to v1 compatibility, so if you are a C# developer then please reach out by ticket or on GitHub.

Audit logging​

To help our partners meet their compliance obligations, we’ve added advanced audit logging capabilities into Comet Server. Almost every action – including login events for administrators and users, restoring data, changes to user and server settings, and more – can now be logged server-side into a secure file.

This work was based on expanding Comet Server’s live event streaming features to cover a much more comprehensive set of data. If you are making use of the existing Webhook feature or the Websocket API feature, when upgrading to Comet Server 23.3.7 or later, you may start receiving a larger amount of data. You can control this by declaring a restriction on the streamable event types you want to receive.

In current versions of Comet, this feature must be configured by setting the AuditFileOptions property inside the cometd.cfg file. We expect to make it available via the Comet Server web interface soon!

Quality, performance, and memory enhancements​

As we approached the quarterly release, we took a special focus on quality improvements to give us a solid foundation.

The fresh new design for the Comet Server web interface had its debut in 23.3.3, and since then throughout the 23.3.x Voyager release track, we have spent additional time and effort to ensure quality. In the latest versions of Comet Server you will see fewer cosmetic issues as this work has stabilised nicely.

We’ve also made significant performance improvements to key areas of the client application. Performing granular restore of single files from inside a Disk Image or Hyper-V backup may now be 3-8x faster while using 40% less memory. Comet also now uses less memory for loading files from S3-compatible Storage destinations like Wasabi, and when loading very large files from a Storage Vault.

The Office 365 Protected Item type in Comet has been a continued development focus this quarter, including a partial rewrite. Comet is now better able to handle incremental changes to mailboxes; the reported error messages are more detailed; and Office 365 backup jobs taken with the latest version of Comet should be more resilient against network errors and rate-limiting from the Microsoft Graph API. We would encourage all users to raise a support ticket if you experience any issues with Office 365 backups as we continue to improve the sync capabilities.

Comet 23.5 “Thebe”​

On the 1st of June, we put the finishing touches on our latest quarterly release, Comet 23.5.0 “Thebe”.

Thebe is the latest entry in our quarterly rollup series. It branches off from our main rolling Voyager development into a fixed target for you to qualify and build your service offering upon.

Like Comet’s previous recent quarterly software releases, “Thebe” is named after a moon of Jupiter, which in turn takes its name from an ancient Greek mythological figure. It is a very small moon with only 0.004x Earth’s gravity, meaning it is less likely to shape itself into a sphere – in fact a large impact crater covers around 40% of its surface area:

It was discovered by the Voyager 1 and 2 spacecraft in 1979, but owing to its small size and position, almost nothing more could be discovered until the next spacecraft Galileo visited two decades later. Scientists believe it contains frozen water.

As always for a new quarterly release, there are two changelogs for 23.5.0 “Thebe” depending on whether you are coming from the previous quarterly release or the previous Voyager release. You can see the full details in the release notes.

That’s all for this month! Thanks for reading – there are some more great features currently under development that we’re excited to be able to share with you soon. As always, please follow @CometBackup on Twitter and you can always contact us if you have any questions.

Changes compared to 23.3.8

Bug Fixes​

• Fix an issue with macOS devices failing to log in with an ERR_UNKNOWN_DEVICE error
• Fix an issue with the Recent Activity and Policies pages not loading correctly when hard-refreshed or loaded directly via URL
• Fix an issue with Comet default branding not presenting correctly following a server upgrade

“What’s New?” is a series of blog posts covering recent changes to Comet in more detail. This article covers the latest changes in Comet Voyager over April 2023.

There were five Comet software releases during April – four in the weekly 23.3.x Voyager release series, plus one minor patch update 23.2.2 for our quarterly “Leda” release track. We’ve also released a new YouTube tutorial for the new Object Lock feature! 

There were some very large and exciting features released in Voyager during April:

New design for the Comet Server web interface​

When upgrading to Comet Server 23.3.3, you’ll be greeted with a new experience:

This is the first major Comet Server web interface upgrade in six years. We’re very excited and proud of the new layout. The left-aligned navigation bar allows for faster navigation to pages without clicking through menus, and the quick search bar has been modernized. The new design has also expanded the set of colour customization options that are available: in addition to setting your custom brand colour, you can also set an accent colour for highlights.

The homepage has seen the most dramatic change, including new navigation buttons on the top-right and a rework of all the admin widgets. There are new widgets showing how many of each Protected Item type is being used; how much used/free space there is on your Storage Role data location; the status and last run time of your Server Self-Backup; a live real-time chart of replication progress; and more.

The policies page has also been redesigned. As we add more and more policy options in new versions of Comet, we split the long Policy section to use sub-tabs. This includes a summary page, and a new feature to suggest possible common file path exclusions. We’re continuing to work on additional Policy features, and you can expect to see a highly-voted feature coming soon!

When looking at a user account, on the Protected Items tab, the new user interface design has also added a quick-access “Run Backup” button that can remotely command the device to start a backup job. Previously, this feature was available from the Connected Devices page or from the Devices tab – but adding it to the Protected Items tab is a significantly more convenient place, and demonstrates this functionality more clearly to new Comet administrators.

We would appreciate hearing your feedback on the new web interface design before it lands in the upcoming quarterly release!

Search restore from web​

When restoring data, Comet prompts you for the Storage Vault to restore from; the backup snapshot to browse inside; and then the file (or all files), respectively. However if you’re trying to restore a single file without knowing exactly when it was last available, or what folder it was inside, Comet’s Search button can search through all backup job snapshots to find the right match.

The Search button has been available on the Restore dialog in the desktop app for a while. New in Comet 23.3.4 is the ability to remotely perform a file search for restore from the Comet Server web interface.

Test Connection for Storage Templates​

Comet is highly flexible in the number of ways you can configure your storage. From the customer’s device running Comet Backup, a Storage Vault could point to a local path; directly at a cloud storage provider; or to your Comet Server with Storage Role enabled – which could then receive the data and store it on a local RAID array or forward it to another cloud storage provider.

Storage Templates are the provisioning system for new Storage Vaults. If you set up a Storage Template for Wasabi or Backblaze, you gain the ability to provision private, per-customer cloud buckets and access credentials with a single click. If you enable Storage Role for receiving data into your Comet Server (or another clustered Comet Server), using a Storage Template can help to very easily provision new Storage Role buckets for each customer.

Comet has long supported a Test Connection button on the Storage Vault page, to check that your custom entered credentials are valid. But when setting up a Storage Template for the first time, the only way to verify that everything was functioning was to attempt to provision a new Storage Vault for a test customer.

In Comet 23.3.5, a new Test Connection button was added to the Storage Template configuration popup in the Comet Server web interface. This allows you to quickly verify that your template is working as expected.

Self-Backup​

Comet Backup requires a connection to a Comet Server to safely store its configuration. But if you are self-hosting the Comet Server application, the Comet Server also should be backed up to mitigate against the risk of data loss. However, you can’t really use Comet Backup for this purpose, since this creates a circular dependency during recovery.

As a solution, Comet Server includes the Server Self-Backup feature. This creates a consistent snapshot of Comet’s configuration files, and allows you to store it encrypted on any supported storage location, including cloud storage. Multiple targets, custom scheduling, and data retention policies are all supported. The files are simple zip files to ensure that any eventual necessary restore is an easy and low-stress process.

The latest version of Comet Server made improvements to the Server Self-Backup feature. The generated filenames now clearly show the date and time of the backup job, instead of solely an epoch timestamp. Any automatic SSL certificate files provisioned by Let’s Encrypt are now included in the archive, ensuring that it is not required to reissue the certificate. This helps avoid any issues with rate limits on the Let’s Encrypt service, which could otherwise prolong your service outage.

We’ve also added a new option to include server log files in the Self-Backup archive. These log files are not generally required, but for completeness or for an investigation, they can provide an additional view into the circumstances behind the event.

Codesigned uninstallers​

Microsoft, along with third-party security vendors, continue to harden the security posture of the Windows operating system. Comet Backup’s client installer is codesigned – either by our company, or if you are using custom branding, then possibly with your own custom codesigning certificate. However over time, the security hardening has increased, and we’ve recently heard reports that the uninstaller for Comet Backup could trigger alerts in some security products. As a result, the latest versions of Comet Server apply Authenticode codesigning to the uninstaller to help avoid this issue.

The 23.5 Quarterly release is coming soon​

At Comet, we release our software under two tracks – the “Voyager” release track approximately weekly, with all of our very latest changes; and the quarterly release track, where we bundle up three months’ worth of development into a new fixed point for you to qualify, offer, and build upon, in order to provide a consistent experience for your own customers. Depending on your market position or your requirements, you may find either one of these tracks better suits your needs. As per our regular release schedule, you can expect a 23.5.0 quarterly release towards the end of this month, which will bring all of the exciting features to the quarterly track and also for Comet-Hosted users.

That’s all for this month! Thanks for reading – there are some more great features currently under development that we’re excited to be able to share with you soon. As always, please follow @CometBackup on Twitter and you can always contact us if you have any questions.