Supply Chain

The ConnectWise (SkyKick) Breach: Investigating a Compromised Vendor Service Principal

On October 15, 2026, a credential belonging to the ConnectWise Cloud Backup application (formerly SkyKick Cloud Backup) was used to mint Microsoft Graph tokens inside customer Microsoft 365 tenants. Huntress confirmed unauthorized sign-ins in 162 tenants as of 19:00 UTC on October 16, and reports sign-in activity from the attacker's address against nearly 1,000 Entra tenants, over 300 of them successful. Treat the confirmed number as a floor, not a scope. The credential lives in the vendor's tenant, not yours, so nothing a customer did or failed to do produced it.

This is a workload identity compromise, and it breaks most of the muscle memory a SOC has for account takeover. There is no user to disable, no password to reset, no MFA prompt that could have fired, and no risky sign-in to triage. What there is happens to be exactly the material an investigation needs: the application object, the service principal that instantiates it in your tenant, the permissions you consented to years ago, and a directory audit trail of everything that identity changed. This post walks that evidence in the order you should pull it, labelling it with the same evidence tiers any identity investigation uses, and shows how Cade's Microsoft connector assembles it in one pass instead of eleven portal blades.

What happened

A vendor application credential (a client assertion) for a backup product that MSPs install across every tenant they manage ended up in an attacker's hands, and the attacker used it the way the product uses it: with full consent and no friction. The sequence Huntress reconstructed:

When (UTC) Activity
Oct 13 to 14 Password spray against four law-firm tenants from 144.172.64[.]128, user agent curl/7.88.1. Smart lockout caught two accounts. No successful user sign-ins observed.
Oct 15, from 01:58 In roughly 50 monitored tenants, ConnectWise Cloud Management service principals assigned themselves directory roles, mostly Exchange Administrator. User agent kiota-dotnet.
Oct 15, 03:44 to 03:49 In one tenant, the Teams Cloud Management app added itself to Global Administrator, then a Python-urllib/3.12 script added client secrets to two other applications, including a security vendor's own app. The new secrets expire in 2028.
Oct 15, 06:13 to 08:34 Graph token issuance from 144.172.64[.]128 across tenants in sorted GUID order, seconds apart. A scripted run against a target list.
Oct 15, 09:25 to 12:35 Second wave against 16 further tenants, sweeping for Graph tokens. The attempts failed wherever the service principal was not present.
Oct 15, 13:54 to 14:20 An apparently hands-on-keyboard pass reaching for Exchange Online, SharePoint Online, Teams, Azure Resource Manager, Entra, and the Office 365 Management APIs.
Oct 17 ConnectWise has paused Cloud Backup application access while it investigates. It has not said how the credential was obtained, and reports no evidence that partner backup data was taken.

The applications to act on, all owned by ConnectWise tenant 3f9cd7a0-127a-4ed4-bf06-2f830d5616b7. In older tenants these may still carry SkyKick names:

Application Application ID Status
ConnectWise Cloud Backup 506e08f9-6e9a-4da4-942b-a099a03644ae The compromised credential, and the source of the Graph token activity
Cloud Management - Exchange Online b78f6a77-d4e5-46fb-87e5-d1648bd02519 High-confidence abuse, self-assigned directory roles
Cloud Management - MS Teams 24205314-2b89-4e07-a2a4-579e898a5640 High-confidence abuse, self-assigned Global Administrator in at least one tenant
Cloud Management - SharePoint Online b2b55f5b-7758-4719-8d8f-88ae172e6bdb High-confidence abuse

These four are where confirmed abuse sits, so they are where to start. They are not the whole inventory: further ConnectWise Cloud Management applications share the same owner tenant, and Cloud Management - MS Graph (0de14c77-b9b0-4137-9360-34878a929827) has also been reported self-assigning directory roles. Cloud Management - Partner Center (412dcf96-176d-4b3a-a1b0-4981ad66971e) and the parent Cloud Management app (2399d300-6fca-4e6a-a686-399fefe75b83) round out the set. Match on the application ID, never the name: "MS Graph" also name-matches the first-party Microsoft Graph service principal (00000003-0000-0000-c000-000000000000) that every tenant has and none should disable. Treat every application that owner tenant returns as in scope for the checks below.

Supporting indicators are weak on their own and should be used to confirm, never to scope: 144.172.64[.]128 (AS14956, RouterHosting), and the user agents curl/7.88.1, Python-urllib/3.12, python-requests/2.31, and kiota-dotnet. The last one is Microsoft's own Kiota-generated Graph SDK, used by ConnectWise's apps and a great many others, so it fires constantly on legitimate traffic. Do not excuse an event because it carries that agent: the post's own timeline pins the Exchange Administrator self-assignments to it. What carries signal is the combination of actor, operation, and agent, never any single value.

Pulling the app and service principal metadata

Start by establishing what actually exists in the tenant, because the answer is frequently "more than the MSP remembers installing." The distinction that trips people up: an application is the registration object, owned here by ConnectWise's tenant; a service principal is the local instantiation in yours, and it is the service principal that holds your consent, your role assignments, and your enabled or disabled state.

Ask Cade:

List the service principals in this tenant matching ConnectWise and SkyKick, and for each one give me the app ID, the app owner organization, whether it is enabled, and when it was created. Flag any whose owner organization is 3f9cd7a0-127a-4ed4-bf06-2f830d5616b7.

Search by name to find the candidates, then confirm each one by GUID: the display names changed across the SkyKick and ConnectWise eras and a renamed app will not match the string you expect, so treat a clean name search as the start of the check rather than the end of it. The four application IDs above and the owner tenant are what actually settle it.

What matters in the answer:

  • appOwnerOrganizationId is what settles whether a hit belongs to this incident. Display names changed across the SkyKick and ConnectWise eras, so a name search finds candidates but can never rule an app out. For the authoritative list, query Graph directly: GET /v1.0/servicePrincipals?$filter=appOwnerOrganizationId eq '3f9cd7a0-127a-4ed4-bf06-2f830d5616b7', and page it to the end. Reconcile that against what the name search returned, and investigate anything that appears in one and not the other.
  • accountEnabled tells you whether containment has already happened in this tenant or is still owed.
  • createdDateTime dates the original consent, and with it how long the standing access has existed.

Flag any service principal you cannot account for, especially one created in the last week. Across a managed estate this is the step that benefits most from an assistant: the same question has to be answered identically for every tenant, and the answers only mean something side by side. The tenant where the app is absent is fine; the tenant where it was created last Tuesday is not.

Proving what permissions it held

This is the question every executive asks within the hour, and the one most teams answer from memory. Do not. Pull it.

Give me the full detail on the ConnectWise Cloud Backup service principal, including every app role assignment, and list the delegated OAuth2 permission grants attached to it.

Two distinct permission models, and you need both. Within the first, a short list deserves checking by name:

  • Application permissions (app role assignments) are the dangerous ones here. They are tenant-wide, require no signed-in user, and are exactly what a client-credentials token carries. Mail.ReadWrite, Files.ReadWrite.All, Sites.FullControl.All, and Directory.ReadWrite.All are the kinds of grants a backup product asks for, and any one of them describes a serious incident.
  • Three permissions in particular turn a data breach into a tenant takeover. RoleManagement.ReadWrite.Directory is the one that lets a service principal add itself to a directory role, which is the self-elevation observed in the wild here; AppRoleAssignment.ReadWrite.All lets it grant itself any other permission it wants; and Application.ReadWrite.All lets it write credentials onto other applications. Expect to do one translation step to find them: an app role assignment names its permission by appRoleId, a GUID, not by the string you are looking for. Resolve the mapping from the authoritative source rather than a blog's copy of it, including this one, by reading the appRoles collection on the Microsoft Graph service principal (00000003-0000-0000-c000-000000000000) in your own tenant. Directory.ReadWrite.All is broad and serious, but on its own it does not confer privileged role management, so do not read its absence as the all-clear.
  • Delegated permissions (OAuth2 grants) act on behalf of a user and are less directly abusable with a stolen app secret, but they map the rest of the surface and occasionally surface a consent nobody intended to give.

Then invert the question and ask what the identity became rather than what it was given:

List the directory roles in this tenant and their members. Flag any service principal holding Global Administrator, Privileged Authentication Administrator, Exchange Administrator, or Application Administrator.

A service principal sitting in Exchange Administrator is the fingerprint of this campaign. A service principal in Global Administrator is a tenant-level emergency. Neither is something a backup product should ever hold, which makes this one of the rare checks where any hit is a finding.

Reconstructing the related activity

Metadata tells you the potential. The directory audit log tells you what was exercised. Four activities carry this investigation, all of them Observed-tier evidence, and all four under the ApplicationManagement and RoleManagement categories:

Audit activity What it means here
Add member to role A ConnectWise or SkyKick service principal elevating itself.
Add app role assignment to service principal The quieter path to the same place: the app granting itself a Graph permission rather than a directory role. An audit scoped to role changes alone returns nothing on this.
Add service principal credentials A secret or certificate added to any application. This is the persistence that survives the vendor's credential reset.
Update application Changes to an app object: reply URLs, credentials, permission requests. The failed attempts often sit next to the successful ones.

Pull the directory audit logs for the last 14 days in the ApplicationManagement and RoleManagement categories. Show me every "Add member to role", "Add app role assignment to service principal", "Add service principal credentials", "Update application", and consent-grant event, with the initiating actor, the target, and the result.

Two things to watch before you trust what comes back. Both of these listings return a single page, so on a large tenant a 14-day audit pull or a full app-registration sweep can truncate: narrow the window or the category and run it in slices rather than reading a short result as a complete one. And check your retention. Entra ID Free keeps directory audit logs for seven days; P1 and P2 keep thirty. The same seven days applies to the service principal sign-in log, which is where the attacker-IP evidence above lives, so both halves of the record age out together. On Free, seven days back from publication is October 10, so the whole campaign is still in range, with about three days of headroom before October 13 falls out. That is enough time and not a day more: export the results now rather than planning to come back to them, and once the window has rolled past, read an empty result as a gap in your evidence rather than as a finding.

Read the failures as carefully as the successes. In the tenant Huntress detailed, nine changes were attempted against two applications and three failed, including a certificate credential rejected on a thumbprint mismatch before the actor fell back to client secrets. The failures are what reveal intent and tooling; a report listing only successful operations understates what was attempted.

Two caveats belong in the report itself. App-only audit events carry no source IP, so you cannot tie a role self-assignment to 144.172.64[.]128 from the audit log alone. That IP evidence lives in the service principal sign-in log, which in the Entra portal is a separate tab from the user sign-ins your analysts default to, and filtering that tab by the application IDs above is the step most tenants skip. Do it there, and bring the result back as the IP-level corroboration the audit trail cannot give you. And the directory audit log covers directory changes only; for mailbox and file activity you need the Microsoft 365 unified audit log and Defender advanced hunting, where Cade can run CloudAppEvents queries over the same window.

Then widen past ConnectWise:

List the app registrations in this tenant with their credential counts and next expiry. Flag any whose credential count I would not expect, especially on management or security tooling.

Cade returns credential counts and expiries with the secret material stripped, so the answer is safe to paste into a ticket. Note what this sweep can and cannot see: an application you registered has an application object in your tenant and shows up here, but a vendor app owned by someone else's tenant exists in yours only as a service principal, and a credential planted on that service principal will not appear in an app-registration listing. For those, the Add service principal credentials events from the audit query above are the reliable record. You are hunting for a credential you did not create, on an app you do trust. Resist the temptation to hunt on the expiry date: a two-year life looks suspicious until you remember it is the Graph default and the portal maximum, so the attacker's 2028 secret is indistinguishable from every legitimate one. When the credential was added is the discriminator. Graph carries it: every entry under passwordCredentials and keyCredentials has a startDateTime, so pull those directly and sort on it. Do not trust it on its own, though. That field is caller-supplied on addPassword and on a PATCH, so an actor holding Application.ReadWrite.All can backdate a planted secret to sit innocently among the old ones. It is a fast way to find the obvious plants, not a way to clear anything. Cade's listing strips the credential objects down to counts so nothing sensitive lands in a ticket, which means the count is your tripwire and the start date is the confirmation. The Add service principal credentials and Update application audit events corroborate both, and matter most once the audit window starts to expire.

Containment and remediation

Order matters, and the first two steps are commonly done backwards.

  1. Disable the service principals. Do not delete them. Set the ConnectWise Cloud Backup app and every Cloud Management app present to disabled. Deleting removes the consent record and the audit linkage, destroys evidence you will want in two days, and forces a full re-consent to restore service later. Cade can disable a service principal directly through the connector, which keeps the action and its justification in the same audit trail as the investigation that prompted it. Disabling does not invalidate a token already issued: an access token minted a minute earlier stays good until it expires, typically up to about ninety minutes. Treat that window as live and work the rest of this list inside it, watching for anything the identity does on its way out.
  2. Remove attacker-added credentials everywhere, not just on ConnectWise apps. Any secret or certificate added since October 13 that you cannot attribute. Until this is done, the rest of the remediation is theater. Like step 3, this one needs a second pass after the token window closes: an identity holding Application.ReadWrite.All can plant a fresh secret on its way out, and the one you removed first is not necessarily the last one there.
  3. Strip the directory roles. Remove any role a ConnectWise or SkyKick service principal holds that you did not deliberately grant. Do this after disabling, but re-check it once the token window above has closed: inside that window a live token can still put a role back, so the first pass is containment and the second is the one you record.
  4. Revoke delegated grants you no longer need. If the product was shelfware in this tenant, revoke the consent outright rather than leave a dormant grant for the next administrator who wonders why backups stopped.
  5. Do not expect Conditional Access to have helped, and do not buy it expecting it to. Ordinary Conditional Access targets users and is never evaluated on an app-only client-credentials request, so an exclusion naming these apps neither widened this attack nor closes it. Conditional Access for workload identities is the obvious next thought and it does not reach this case either: it can only target single-tenant service principals registered in your own tenant, and a vendor app owned by someone else's tenant cannot be assigned to a policy at all. There was no Conditional Access posture that would have stopped this. What does bite are application management policies, which can cap credential lifetimes and block secret-based authentication tenant-wide, and keeping Application Administrator and Cloud Application Administrator down to a handful of people. Clear out stale user-policy exclusions on the same pass for the attacks where they do count, but do not log it as the fix for this one.
  6. Hunt the follow-on. Tokens with mailbox scope lead where they always lead. Check for new inbox and forwarding rules, anomalous mail access, and newly consented OAuth applications across the affected window. This is the BEC investigation path, and Cade works it through the same connector: enumerate inbox rules per mailbox, disable the ones that should not be there, and revoke sessions where a user identity was also involved.
  7. Keep the apps disabled until ConnectWise confirms every credential has been reset. Expect a backup gap, and tell the customer about it before they find it.

One judgment call to make explicitly rather than by default: a clean service principal sign-in log does not clear the tenant. Coverage depends on licensing and retention, the audit log never recorded the reads, and the first wave ran against a target list nobody has published. Disable the apps regardless of what you find, and write the report as "no evidence of access observed" rather than "no access occurred." Those are different claims and only one of them is defensible.

Takeaway

The hard part of a vendor service principal compromise is not any single query. It is that the answer lives in six places that do not agree on a schema, and the whole set has to be re-pulled for every tenant in the estate before anyone can say what the exposure was. For an MSP that is a day of portal work per tenant, done under pressure, by analysts who will reasonably skip the step that matters most: looking for a planted secret on an application that is not ConnectWise.

That consolidation is the work Cade's Microsoft connector is built for. One question, asked in plain language, pulls the app and service principal metadata, the effective permissions, the role assignments, and the audit trail together, separates what was Observed from what is Assessed, and carries the containment actions back through the same audited path. The vendor's credential was never yours to protect. What happened next in your tenant is, and the record of it is sitting in Graph waiting to be asked for.

Incident details in this post are drawn from Huntress, SpecterOps, and BleepingComputer as of October 17, 2026. This is an active incident; verify the current status and the affected application list with ConnectWise before acting.