Changes compared to 23.6.11 

Enhancements​

• Improved performance of cleaning up Object Lock delete markers during retention passes for S3-compatible Storage Vaults
• Improve initial startup performance of browsing files to restore from a disk image

Bug Fixes​

• Fixed an issue with errors during a retention pass on an S3-compatible Storage Vault causing the entire retention pass to hang indefinitely
• Fixed an issue with missing JPEG image format support for custom branding images in the Comet Backup desktop app on Windows
• Fixed an issue with counting effective group membership for Office 365 Protected Items
• Fixed an issue with Microsoft 365 batch retry logic not using fallback method when limit is reached
• Fixed an issue with a crash when browsing files from a Hyper-V backup
• Fixed an issue with Hyper-V backup where browsing file(s) at the time of restore could crash if a directory name ended in “.vhdx”
• Fixed a cosmetic issue with the Comet Server web interface homepage widgets if Comet Server Storage Role is backed by unlimited cloud storage
• Fixed a cosmetic issue with padding around the restore queue section when performing a Disk Image restore from the Comet Server web interface
• Fixed a cosmetic issue with some missing language translation strings
• Fixed a cosmetic issue with incorrect spelling for some WebDAV fields

Changes compared to 23.6.9

New Features​

• New design for the Comet Backup desktop app.

“What’s New?” is a series of blog posts covering recent changes to Comet in more detail. This article covers the latest changes in Comet Voyager over August 2023.

There were six Comet software releases during August – five releases in the 23.6.x Voyager release series, plus one release for the launch of our new quarterly 23.8.x Adrastea release series.

New design for the Comet Backup desktop app​

We’re very pleased to announce a visual refresh for the Comet Backup desktop app:

This work followed on from the new design for the Comet Server web interface that was released earlier this year for the previous 23.5 quarterly series. This is the first major layout change to the desktop app in Comet’s history; the current desktop app has had the same layout for the past six years.

Our goal for the new interface design was to make the product simpler for new users, while still keeping it familiar for existing users. The tabs, icons, buttons, fonts and colors have all been refreshed with a modern rounded style, but you are still greeted with your familiar Protected Items and charts on the home screen.

The most fundamental change was made to navigation within the app: the previous left-side navigation bar has been removed, meaning your Protected Items are always clearly visible, and we have refreshed the breadcrumb bar appearance to help orient you to this landmark interface element. The removed left-side navigation bar has been replaced with a top navigation bar which puts more emphasis on your branded company logo, and offers clearer, more distinctive focus on the calls-to-action for the key backup and restore functionality.

In the top-right corner, you will see a a dropdown overflow menu, using modern and intuitive UX iconography. Inside this menu, you can get quick access to detailed job history; creating recovery media for restoring Disk Image backups; and a new Settings dialog. The new Settings dialog contains Storage Vault management, account and login settings, email reporting settings, your devices, and the ability to import settings from other supported backup products.

If you had configured a custom Help webpage for embedded use, it will now appear as a button on the main screen.

VMware beta program​

We’ve been hard at work developing the next Protected Item type for Comet, to add VMware support.

This has been a highly upvoted feature on our Feature Request page for some time. We’ve spent extra time and effort on making this Protected Item type as polished and performant as possible.

The details are subject to change before release, but we expect to be able to support major versions of VMware ESXi (6 / 7 / 8) and vCenter, using either free or paid licenses. Your backup jobs can be accelerated using changed block tracking (CBT) to produce synthetic full disk images that are deduplicated inside your Storage Vault. The new Protected Item type will work seamlessly with Comet’s scheduling, deduplication, compression, encryption, job reporting, tenants, and granular restore of single files from supported virtual disk filesystems.

We’re in the final stages of the beta program and are currently accepting new partners to help us ensure that this new Protected Item type is a good fit for your production VMware infrastructure. If you are interested in getting early access to this feature, please follow this link to register your interest – we would greatly appreciate any feedback you might be able to give us before the official launch later this year.

Price change notice for Backblaze B2​

Comet has supported Backblaze B2 as a storage platform for over six years, since our 17.6.4 release back in July 2017. Over the years Backblaze B2 has proven to be a reliable, trustworthy, performant, and cost-effective solution. Together with Wasabi, these are the two most popular cloud storage providers amongst Comet users.

This month, Backblaze B2 have announced a price change. The base storage cost is increasing from $5 USD / TB to $6 USD / TB, but egress bandwidth costs are being reduced. For full details, please see their official announcement.

WebDAV​

WebDAV is a storage protocol like FTP, SFTP, or the S3-compatible protocol, that can be used to store files in a remote location. It’s based on HTTP technology and supports password based authentication, as well as transport layer security over HTTPS. The protocol has been around since 1996 and was standardized by the IETF in RFC 4918.

Accessing a remote WebDAV server is a built-in feature of Windows Explorer, as well as macOS Finder and the KDE and GNOME file managers.

Because the capability for accessing remote storage is built into the operating system, WebDAV is simple to use with a very low barrier to entry, helping it maintain a broad user base amongst enterprises, universities, and commercial service providers including Hetzner Storage Box, DreamHost, Yandex Disk, pCloud, and many others.

You can easily host your own WebDAV storage server as the protocol is built into NextCloud, OwnCloud, and the Apache and Microsoft IIS web servers. In particular, users of Synology NAS devices can install the WebDAV Server app from the Synology Package Center for a more reliable alternative than configuring SMB credentials.

WebDAV is available as a storage type for Storage Vaults and for Comet Server Storage Role in 23.6.9 and later.

Quick feedback​

The next time you visit the account.cometbackup.com dashboard, you might notice a new “Feedback” tab on the right-hand side of the screen:

Clicking the “Feedback” text will open a short survey asking for any short thoughts you have about your impressions of Comet and how easy our product is to use. Our Customer Success team would really appreciate any answer you give. After submitting feedback, the tab will disappear, but you can submit more feedback at any time by clicking the “Give us Feedback” link in the page footer area.

If you have long feedback or any questions, we would appreciate this via the existing communication channels, such as a support ticket or an email.

23.8.0 Adrastea​

Earlier this week, we put the finishing touches on our latest quarterly release, Comet 23.8.0 Adrastea. This is the the latest entry in our quarterly rollup series, that branches off from our main rolling Voyager development into a fixed target for you to qualify and build your service offering upon.

As with all our recent quarterly release series, Adrastea is named after a moon of Jupiter, which in turn takes its name from an ancient Greek mythological figure. It is the second-closest moon to Jupiter and the smallest of the four inner moons, orbiting at the edge of Jupiter’s main ring. It is thought to be the main contributor of material to the rings of Jupiter.

For users coming from the previous 23.5 Thebe quarterly release series, Adrastea adds 7 features and 24 enhancements, including the new Comet Backup desktop app design and WebDAV support mentioned above; single sign-on support with OpenID Connect (OIDC); Protected Items that can stay linked with the user’s Policy; additional admin permission options; and many performance improvements.

The 23.8.0 series does remove support for some old versions of macOS. If you have users with old Mac machines that are not able to upgrade the OS, the previous 23.5.x Comet Backup client will remain capable and working when connected to a 23.8.x Comet Server.

The full set of changes can be found in the release notes.

Webinar announcement​

If you’d prefer to watch rather than read, we’re hosting a webinar next week to discuss this new quarterly release and all the new changes. Please register for a notification before we go live on September 5th (5pm EDT / 2pm PDT) to catch up on all the latest Comet news with me – and as usual, there will be time for a live question-and-answer session at the end of the presentation.

As well as that, we have many more videos available on our YouTube channel, including guides on getting started with Comet, individual features, demonstrations with our technology partners, and webinars for previous quarterly software releases.

New Features​

• Added support for performing backups to a WebDAV based Storage Vault

Enhancements​

• Added support for macOS client notarization using notarytool (Xcode 13 or later) with a fallback to altool if not installed. Note: Apple’s notary service will not accept uploads from altool or Xcode 13 or earlier starting November 1, 2023. See this link for details
• Added the ability to verify if versioning is enabled on the bucket when setting up Object Lock on supported S3-backed Storage Templates
• Improved performance for accessing Storj.io Storage Vaults
• Improved the documented types in the PHP SDK

Bug Fixes​

• Fixed an issue with WebAuthn logins for non-OIDC admins failing in the Comet Server web interface
• Fixed an issue with jobs not appearing when clicking on segments on the Recent Activity page of the Comet Server web interface
• Fixed an issue with the Application Aware Writer live picker in the Comet Backup desktop app not accurately displaying what is selected
• Fixed a cosmetic issue with certain characters showing incorrectly in the Microsoft 365 Protected Item dialog in the Comet Server web interface
• Fixed an issue with the Storage Vault “Test Connection” button crashing the Comet Server web interface when Storage Role is disabled

Breaking changes​

• New, simpler configuration may be required for PKCS11 Authenticode codesigning with a hardware dongle

New Features​

• Added support for using OpenID Connect 1.0 for Single Sign-On for Comet Server admin users. This allows admin users to authenticate themselves to Comet Server using their existing credentials at an external Identity Provider, such as Microsoft Entra ID (formerly Azure AD), Google, or other OIDC providers

Changes compared to 23.5.2 

Bug Fixes

• Fixed an issue where Wasabi error logs caused by System Clocks being too far behind or too far ahead not outputting as expected.
• Fixed an issue with macOS notarization timing out when Apple takes longer than 10 minutes to process the request.
• Fixed an issue with Office 365 clients where rate-limiting may cause list requests to restart unnecessarily
• Fixed an issue with the Comet Backup desktop app crashing when loading if a recent backup job was extremely large.
• Fixed an issue with Office 365 backups where workers can hang while querying the Vault
• Fixed an issue with bucket name validation where Wasabi bucket names ending with a period were reported as invalid
• Fixed an issue with error handling during a retention pass of a vault that has object lock enabled.
• Fixed an issue with the Comet Server Docker container failing to start during a recreate event.
• Fixed an issue with Office 365 clients hanging in some situations.

“What’s New?” is a series of blog posts covering recent changes to Comet in more detail. This article covers the latest changes in Comet Voyager over July 2023. 

There were five Comet software releases during July – one in our quarterly 23.5.x Thebe release series, plus four releases in the 23.6.x Voyager release series.

Single sign-on with Microsoft, Google, and OIDC​

Comet Server is adding support for administrators to single sign-on (SSO) to the Comet Server web interface, using a supported OpenID Connect (OIDC) identity provider:

OIDC is a framework for authentication and authorization, based on the OAuth 2.0 standard. It’s widely used by many providers for “Log in with…” buttons. This new feature extends our existing single sign-on support using the LDAP protocol.

The additional identity providers (IdPs) now supported are:

• Microsoft Entra ID (formerly Azure AD)
• Google (Google Cloud, Google Workspace, or personal)
• Any other OIDC-compatible provider that uses a discovery document (usually at the .well-known/openid-configuration URL path).

You can configure a new OIDC provider from the Comet Server web interface > Settings screen > “Admin Accounts” tab > “External Authentication Sources” button:

To use this feature, you should first visit your IdP’s settings page, register a new application credential within the IdP, and copy the credentials to this settings page. You will then need to copy Comet’s generated “Redirect URI” field back into your IdP’s settings page.

When the administrator user uses the new “Log in with…” button and performs a successful login operation via the IdP, a new Comet Server administrator account will be dynamically created for them on-demand. As with LDAP, you can specify which Comet Server permissions are granted to the newly generated account. This new account is marked as “Externally managed” within the “Admin Accounts” table, ensuring that valid IdP login is required to access this administrator account.

If your IdP enforces two-factor authentication (2FA), you can configure Comet Server to skip enforcing its own internal 2FA on the account, so that the user is not bothered twice.

You can request custom scopes, and enforce claim values against either custom scopes or standard OIDC scopes. This allows you to enforce that the only members of certain Microsoft, Google, or OIDC groups within your IdP are allowed to log in to the Comet Server.

This feature is available both for the top-level Comet Server administrator as well as individually for each tenant.

Price change notice for Comet Storage powered by Wasabi​

Our Comet Storage service gives you the option to purchase Wasabi Cloud Storage directly from Comet, offering all-in-one billing and providing a more integrated experience. This month, we’ve passed along the latest price changes from Wasabi, to their new price of $6.99 / TB.

For more information, please see Wasabi’s official announcement.

Improved job start performance​

When backing up a Files and Folders-type Protected Item, one of the first steps is for Comet to enumerate all the selected files, in order to calculate their total size. The total size is used to enforce the “All Protected Items Quota” feature, as well as to properly determine the progress bar’s expected upper bound. If you are running a headless device with no GUI to render the progress bar, and you are not using the “All Protected Items Quota” feature, then there’s no remaining purpose for this scan phase, and so Comet will skip it to save time.

We heard mixed feedback about this – a discussion in our feature voting system uncovered some use-cases where the progress bar would still be desirable even on headless devices with no GUI. But also, there was competing feedback that spending time on file size measurement is still slow and undesirable even in some cases where the GUI was present.

In the latest version of Comet, we’ve come up with a new and better approach to this issue. If the “All Protected Items Quota” feature is used, then we require an accurate measurement up-front regardless. But if this feature is not used, then we can rapidly create an approximate progress bar size based on the previous backup job’s size plus some small estimated buffer amount. This should provide a great speed improvement for GUI users, a reasonable progress bar for headless users, and at the same time, provide an accurate measurement for quota users. The reported size measurement will always be completely accurate after the backup job finishes.

Improved low-memory modes​

For users using Comet on devices with low RAM, our software has long since offered the “Prefer temporary files instead of RAM (slower)” option for backup jobs, to toggle whether Comet stores the deduplication index either in-memory or on-disk in a temporary database file. Enabling this option can significantly reduce Comet’s memory usage, allowing the backup job to complete on low-memory devices, at the expense of a longer backup job duration.

The latest version of Comet extends this option to also use a small in-memory bloom filter. This allows Comet to perform some of the deduplication operations in-memory with minimal overhead. This new combination technique can significantly improve the performance of this option for low-memory devices.

The deduplication index is needed for almost all operations involving the Storage Vault, not just backup operations. This month, we’ve also added an option to use temporary files instead of RAM during a restore, extending the possible use cases for Comet on low-memory devices.

Performance improvements for new servers​

The performance improvements this month are not limited to the Comet app itself. We’ve also significantly improved the account.cometbackup.com system: downloading the large Self-Hosted Comet Server installer is now implemented via an Amazon CloudFront cache, improving download speeds between 2-6x in our testing.

We have also been able to significantly improve the speed of creating new Comet-Hosted Comet Server instances. The creation time has been reduced from 60-90 seconds down to 10-15 seconds, owing primarily to some changes in the default generated DNS names.

Configuration change notice for PKCS11 codesigning​

Comet supports Authenticode codesigning for Windows using either an on-disk file (PKCS #12), or a hardware security module such as a USB device (PKCS #11), or a cloud HSM on Azure Key Vault. With the file-based approach no longer being supported for new Authenticode certificates, we are seeing increased use of the alternative PKCS #11 and Azure Key Vault options, as partner Authenticode certificates come up for renewal.

If you are using a physical USB device for Authenticode codesigning, we have updated the available settings options to improve compatibility with a wider range of devices. The new settings screen should be clearer and easier to use, but you may be required to update your configured settings, as depicted:

If your Comet Server is running in a cloud VM, it’s not feasible to plug in a USB hardware device for codesigning. We would recommend Azure Key Vault as an excellent cloud-based solution to this issue, but we’ve also recently tested the compatibility of the third-party Virtualhere software for remotely forwarding a physical USB device to another PC, and we can confirm this solution works for PKCS#11 codesigning when running Comet Server on a cloud VM.

Quick search​

When using the Comet Server web interface, the quick search bar (using the Alt + Q keyboard shortcut) could previously search through usernames, Account Name field values, Protected Item and Storage Vault names, settings pages, and more. In the latest version of Comet, we have extended the search capabilities to also find users by their email address.

You can also now enter the ID of a Protected Item, Storage Vault, or even a backup job, and the quick search bar will try to match it with the corresponding user or job. This is particularly helpful for troubleshooting some situations.

That’s all for this month – the blog will return next month with more news about all the latest changes to Comet.

New Features

• Added an option to use temporary files instead of RAM during a restore
• Added a partial data recovery option to zero over unreadable data chunks during a restore (Comet will not abandon the job when it finds an unreadable chunk)

Enhancements

• Improved error logging for failures during MSSQL backups and restores

Bug Fixes

• Fixed an issue with Office 365 clients where rate limiting may cause list requests to restart unnecessarily
• Fixed an issue with Office 365 clients hanging in some situations
• Fixed an issue with macOS notarization timing out when Apple takes longer than 10 minutes to process the request
• Fixed an issue with error handling during a retention pass of a Storage Vault that has Object Lock enabled

Changes compared to 23.5.1

Bug Fixes​

• Fixed an issue with the Comet Server session timeout not being respected by the Comet Server web interface.
• Fixed an issue causing Comet Server web interface sessions to terminate unexpectedly on servers without the Authentication Role enabled.

Changes compared to 23.5.0

Enhancements​

• Automatically wait for settings changes to be applied in the Comet Server web interface

Bug Fixes​

• Fixed an issue when upgrading the Comet Client which caused the client to logout after upgrade
• Fixed an issue with server self-backups failing if the autossl, resources, or logs subdirectories are not present
• Fixed an issue related to error handling in Office 365 backup
• Fixed an issue with Office 365 client where in certain scenarios, the client may lock up
• Fixed an issue related to Scaleway Object Storage and Synology C2 Object Storage failing to connect
• Fixed an issue with high memory usage during retention passes, even if the “Prefer temporary files instead of RAM” option was set
• Fixed an issue with disk image browsing where MBR partitioned disk might cause a crash
• Fixed an issue with snapshot IDs not being shown in the Comet Server web interface restore dialog when advanced options are enabled
• Fixed an issue with the event stream websocket not staying open after authentication
• Fixed an issue with the SEVT_META_HELLO event missing the TypeString field
• Fixed an issue with high Comet Server Auth Role memory usage if there are a large number of live connected devices
• Fixed an issue with the Policy summary showing job delay in seconds instead of minutes
• Fixed a cosmetic issue with server page content rendering before the navigation on first load
• Fixed a cosmetic issue with the server navigation logo flashing following certain actions
• Fixed a cosmetic issue with unexpected horizontal scrollbars appearing for some tables in the server interface
• Fixed a cosmetic issue with some active pages not being highlighted in the navigation
• Fixed a cosmetic issue with exposing Wasabi hostnames in white-label error messages.