Skip to content

Conversation

@mikehardy
Copy link
Collaborator

@mikehardy mikehardy commented Jan 29, 2026

Description

Our Simulator runtimes were just deprecated and removed: actions/runner-images#13570

Also, there are intermittent failures on iOS runners related to the build failing which seem to be related to simulators not being consistently available. Works most of the time, fails sometimes.

Per discussion in the upstream repo as linked in the comment in this diff, simply listing the simulators as a step before the ones that fail may be a viable workaround, although there are many indications that it does not 100% fix the issue.

Notes:
1- we may need to download the runtime but this may apparently add a lot of CI time actions/runner-images#12758 (comment)
2- it may be that the runtime already exists but is simply not loaded, and reloading it may be done without adding too much CI time - https://github.com/getsentry/sentry-cocoa/pull/6837/changes#diff-0e892ebad2a2b4f630bce02ab66a0a7ead71801904e24dd1cc30b94ce907e598
3- it may be that we still need to create the simulator even then - example https://github.com/siteline/swiftui-introspect/pull/482/changes
4- it may be that switching from a -destination <simulator name> in build to a -target <iphoneos> and using any simulator would work best though there are reports that there are zero simulators available at times [edit: we already do this]

Of those options,

  • 1 appears to be overkill but is also easy to implement
  • 2 appears to be supported by evidence and the most lightweight solution. It's in use by Sentry. But it only helps if the runtime already exists, and in our case it does not
  • 3 appears unnecessary if we know a simulator is supposed to exist based on installed software and only intermittent failure.
  • 4 is done but doesn't help,

So, option 1 it is - I download the runtime now if it is not available. I skip the download if it isn't.

This adds ~5mins to CI but lets us whatever runtime we want.

This is important because the current runtimes - 26.2 - are awful. So slow with performance problems that they flake all the time. 26.0.1 was the last one that performed decently. But in order to use it we have to download the runtime now.

👀 - while I was in there I tweaked coverage as well. It is not useful to have codecov project level stats give us a failing CI run status when we don't gate PRs on that status. So I set it to informational mode. I want to see the coverage, but not have it fail the status checks.
👀 - I also disabled a test that was failing in ios release mode, it will skip now if it is ios + ci. Runs fine locally

There were a couple other tiny workflow tweaks in there that each got their own commit but aren't worth mentioning - timeouts and workflow step labels etc

Related issues

Release Summary

All test

Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
    • Yes
  • My change supports the following platforms;
    • Android
    • iOS
    • Other (macOS, web)
  • My change includes tests;
    • e2e tests added or updated in packages/\*\*/e2e
    • jest tests added or updated in packages/\*\*/__tests__
  • I have updated TypeScript types that are affected by my change.
  • This is a breaking change;
    • Yes
    • No

Test Plan

This was developed while watching the iOS e2e test carefully.

I'll run a 30-iteration flake hammer on it as well to make sure it didn't regress from prior to the runtimes disappearing


Think react-native-firebase is great? Please consider supporting the project with any of the below:

@vercel
Copy link

vercel bot commented Jan 29, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
react-native-firebase Ready Ready Preview, Comment Feb 2, 2026 8:04pm

Request Review

@mikehardy
Copy link
Collaborator Author

Indeed, that did not work.

xcodebuild: error: Found no destinations for the scheme 'testing' and action build.

Will try the runtime reloading script next

@russellwheatley
Copy link
Member

russellwheatley commented Jan 30, 2026

@mikehardy - this worked for my callable stream PR: 9fce0be

edit- I should add that I have only ran once so maybe it's intermittent like existing.

@mikehardy
Copy link
Collaborator Author

@russellwheatley that will help because as I understand it you will now be in sync with the current installed tuples of Xcode/SDK/Simulator-Runtime

But it still may be intermittent on Runtime availability, or on performance of the runtime (the current ones have problems and caused sim boot / timeout flakes - which is why I pinned Xcode to a lower version before).

If we want to stray at all from the recently-shrunk officially supported version windows of the Xcode/SDK/Runtime tuple then we'll have the problem again

I'm looking for something more like what sentry implemented where they can specify arbitrary tuples and it works with no availability flakes

...then hopefully the performance flakes in the newer runtimes are fixed (apparently Apple is working on it, per note from Github runner image maintainers)

It's all just messy.

@mikehardy mikehardy force-pushed the @mikehardy/workaround-ci-simulator-unavailability branch from dff754c to a38df91 Compare January 30, 2026 15:07
@mikehardy mikehardy force-pushed the @mikehardy/workaround-ci-simulator-unavailability branch from a38df91 to 21cf08b Compare January 30, 2026 15:14
@mikehardy mikehardy force-pushed the @mikehardy/workaround-ci-simulator-unavailability branch from 2a8bb92 to 335b64e Compare January 30, 2026 16:41
@mikehardy mikehardy force-pushed the @mikehardy/workaround-ci-simulator-unavailability branch from 53b86b5 to 13d2c22 Compare January 30, 2026 20:54
@mikehardy mikehardy force-pushed the @mikehardy/workaround-ci-simulator-unavailability branch from 13d2c22 to 4829612 Compare January 30, 2026 22:46
@mikehardy mikehardy force-pushed the @mikehardy/workaround-ci-simulator-unavailability branch from 4829612 to eadb033 Compare January 30, 2026 23:54
@mikehardy mikehardy force-pushed the @mikehardy/workaround-ci-simulator-unavailability branch from eadb033 to a28c4d5 Compare January 31, 2026 00:26
github actions runners do not always have the SimRuntime installed
for the Xcode version we select, for disk space reasons

If you are using latest-stable that won't affect you but if you stray
from that version at all it might

Detect this case and install the runtime if needed
simulator performance gets worse with each macOS/iOS version
maintain comments showing coverage, maintain fail on patch coverage
issues, but the whole project fails frequently because uploads don't
send or aren't processed - those have little value at this point
@mikehardy mikehardy force-pushed the @mikehardy/workaround-ci-simulator-unavailability branch from a28c4d5 to 9fcba99 Compare February 2, 2026 17:16
@mikehardy mikehardy force-pushed the @mikehardy/workaround-ci-simulator-unavailability branch from 9fcba99 to c66926a Compare February 2, 2026 18:58
@mikehardy
Copy link
Collaborator Author

I've got the iOS tests passing again, and we need this in quickly to unblock all the other work here, so merging it now
Just re-self-reviewed everything and it looks good to me, will obviously handle any follow-ups as needed / you break it you buy it

@mikehardy mikehardy enabled auto-merge (rebase) February 3, 2026 01:34
@mikehardy mikehardy disabled auto-merge February 3, 2026 01:34
@mikehardy mikehardy merged commit f8c7959 into main Feb 3, 2026
9 of 21 checks passed
@mikehardy mikehardy deleted the @mikehardy/workaround-ci-simulator-unavailability branch February 3, 2026 01:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants