Delivering Unity macOS build to Steam and AppStore

, , , ,

中文 | English

This blog records how to setup and config the application bundle which will be submitted to the AppStore or Steam by using the CLI tools. Mostly, we use XCode for developing iOS and macOS applications, which automatically handles the code signing and configurations. But when exporting an app bundle from Unity Editor, we have to do this work by using command line tools that are provided by Apple.

In the begining, let’s get some preliminary knowledege about the CodeSign and Distributions. Or you can just skip this part then jump into the process parts.

Submit to Steam store or third-party distribution
Submit to macOS AppStore

Bundle Config and Code Signing

Basic Concepts

Apple provides code signing tool which is called codesign. In general, the execuatable application in macOS system presents as a folder that has name with .app suffix. It’s different from windows. So for signing the app become more complicated, because we are facing a bunch of files not just a single .exe file.

Designated Requirement(DR)

Every application requires a Designated Requirement(DR) when code signing. The DR is identified by the bundle’s BundleID.

Resource envelope

CodeSign tool will process all sourcefile and non-sourcefile. Every file calculates a hash string, then combined these hashes into a final hash value which will be attched to the application bundle.

Any modification apply to the signed files will change the hash that belonging to this files, and obviously affects final hash. By using this method, other security programs can verify the identity and integrality.

The concept for files signature data is call Resource envelope.

Resource envelope has two versions, the latest v2 version will check all the files inside the bundle. After OSX Mavericks 10.9, all app use the v2 version.

Even if we didn’t touch the bundle file, any extra file added to the Game.app bundle folder, will casue the bundle codesign verification failed.

Process application bundle

Modifiy the BundleId

Directly open the Plist file Game.app/Contents/Info.plist change the value of key CFBundleIdentifier。

Remove useless files

Unity Edtiro always genereate .meta files for all the imported macOS library. These .meta file were placed at illegal path for application bundle. And causing the codesign process failed. We must delete them.

1
2
#remove all remove meta files
find "./Game.app" -name "*.meta" -type f -exec rm {} \;

CodeSigning

Signing Third-party libs

Apple requires all the libraries must be signed before submiting to AppStore or Notarization.

All libraries include .dylib .framework and .bundle.

When submiting to AppStore, the third-party libs must be signed by current Apple Developer, even if them were codesigned or not. Which means that we also need to codesign the common shared libs such as libssl.dylib.

Entitlements Files

Before use codesign tool, we need to create .entitlements file. Xcode usually create them for us, but now we are using CLI tools and tackling with exported binary file, don’t we? The entitlements file which used for AppStore submission differ from the Steam one.

The minimal configurations are provided below, other key/value pairs must be added if there have other capabilities.

For macOS AppStore

1
2
3
4
5
6
7
8
9
10
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>com.apple.security.app-sandbox</key>
<true/>
<key>com.apple.security.files.user-selected.read-only</key>
<true/>
</dict>
</plist

For Steam

1
2
3
4
5
6
7
8
9
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>com.apple.security.cs.disable-library-validation</key>
        <true/>
        <key>com.apple.security.cs.disable-executable-page-protection</key>
        <true/>
</dict>

CodeSign Tool

codesign command for signing a single file

1
codesign --deep --force --verify --verbose=4 --timestamp --options runtime --entitlements  $EntitlementsFile --sign "$CertName" $FileName

Parameter sign should be filled with the name of certificate. Like Mac Developer:XXXXX(TEAMID). Codesign tool will search all the certs in user’s keychain, find the correct one. Throwing error when found multiple certs match the inputs.

Command for verification the codesign status.

1
codesign --verify --deep --strict --verbose=4 Game.app

Success with these outputs.

1
2
Game.app: valid on disk
Game.app: satisfies its Designated Requirement

Submit to SteamStore and non AppStore distribution

The binary app which submitted to SteamStore must be notarized by Apple, otherwise the program will be blocked by GateKeeper when user try to launch the app.

The specified notarizing docs from Apple Notarizing macOS Software Before Distribution

Generate Certificate

First step is creating a certificate. Signin to the Apple Developer website then generate a Developer ID Application cert.

Only the account holder of the developer team can create this Developer ID Application Certificate ，even if the user have admin permission role. Just like any other developer cert, export a CSR file from keychain tool, then upload for swapping a .cert file.

Bundle preparation

Now we have a Game.app binary bundle, and a Developer ID Application Certificate. But before signing, bundle needs to be preprocessed.

Following the instruction of first part Process-application-bundle.

  1. delete all .meta file recursivly inside the Game.app folder.
  2. Create Game.entitlements file as informed.
  3. Modify Game.app/Contents/Info.plist if needed.

Fix SteamLib

Most of developer use third-party Steamworks Librlabrecque/Steamworks.NET with Unity, because the VALVE didn’t provide the C# version SDK。Before codesign we needs some patches。

The lib file of rlabrecque/Steamworks.NET is Steam_api.bundle.

  1. Steam_api.bundle contains an invalid CFBundleIdentifier: com.rileylabrecque.steam_api. Change it to com.rileylabrecque.steamapi in the file
    steam_api.bundle/Contents/Info.plist。
  2. libsteam_api.dylib contains 32bit symbols. We only use 64bit symbols, delete it with command lipo -remove x86_64 libsteam_api.dylib -o libsteam_api.dylib.

No we have successfully processing the Steamwork library. These rules are also demanded with the other libraries. You should apply the patches if there exist.

CodeSigning

The instruction of apple developer doc recommands that codesign all libs before signing the final bundle. Using --deep parameter does not guarantee the correctness with nested libraries. It’s just for codesign patching after file modification.

I just write a simple bash script for codesign command execution.

  1. Sign all libraries

$BINARY = ‘Game.app’

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
libfile=`find ./$BINARY -name '*.dylib'`
while read -r libname; do
        echo "Sign:" $libname
        codesign --deep --force --verify -vvvv --timestamp --options runtime --entitlements  $ENTITLEMENTS --sign "$SIGNCERT" $libname
done <<< "$libfile"

bundlefile=`find ./$BINARY -name '*.bundle'`
while read -r bundlename; do
        echo "Sign:" $bundlename
        codesign --deep --force --verify -vvvv --timestamp --options runtime --entitlements  $ENTITLEMENTS --sign "$SIGNCERT" $bundlename
done <<< "$bundlefile"

frameworkfile=`find ./$BINARY -name '*.framework'`
while read -r frameworkname; do
        echo "Sign:" $frameworkname
        codesign --deep --force --verify -vvvv --timestamp --options runtime --entitlements  $ENTITLEMENTS --sign "$SIGNCERT" $frameworkname
done <<< "$frameworkfile"
  1. CodeSign final application bundle
1
2
3
codesign --deep --force --verify --verbose --timestamp --options runtime --entitlements  $ENTITLEMENTS --sign "$SIGNCERT" $BINARY -i $BUNDLE_ID
# verification
codesign --verify --deep -vvvv --strict Game.app

Emphasis: --sign parameter will automatically match, just input “Developer ID Application:”.

Submit to Notarization

  1. Zip archive the bundle before notarization

Warning: We should use the command ditto for archive the budnle, use zip cause the missing of signature infomation

1
ditto -c -k --sequesterRsrc --keepParent "Game.app" "Game.app.zip"

2.Submission

Now we have the ziped bundle. It’s time for notarization.

1
2
3
4
xcrun altool --notarize-app --primary-bundle-id "${Bundle_Identifier}" --username "${APPLE_ID_ACCOUNT}" --password "${APPLE_ID_PWD}" --file Game.app.zip --asc-provider "${ASC_PROVIDER}"

#sample
xcrun altool --notarize-app --primary-bundle-id "com.xxxx.game" --username "yourid@gmail.com" --password "password" --file Game.app.zip --asc-provider "ABCDEFGH"

Parameter asc-provider requires ProviderShortname.

It’s commonly equals Team ID which can be found at
‘Apple Developer/Membership/Team ID’.

But unfortunately, my apple developer account is pretty old. These are not equal.
Don’t worry, apple provide the tools for quering the ProviderShortName.

1
$ xcrun altool --list-providers -u "AC_USERNAME" -p "PWD"

If everything gose well, we may get a RequestId like ‘1809fca2-XXXX-XXXX-XXXX-14aa62f6XXXX’.

1
2
altool[16765:378423] No errors uploading 'Game.app.zip'.
RequestUUID = 1809fca2-XXXX-XXXX-XXXX-14aa62f6XXXX

With this ReqId, we can query the result of notarization. It needs about 5mins for apple server processing the notarization. Maybe longer depends on the file size.

1
2
#查询结果
$ xcrun altool --notarization-info "1809fca2-XXXX-XXXX-XXXX-14aa62f6XXXX" --username "${APPLE_ID_ACCOUNT}" --password "${APPLE_ID_PWD}"

When processing finish, Apple will send an email to your developer ID.
If the notarization failed, all erros can be find by check the log file.

  1. Staple App bundle after notarization
1
2
$ xcrun stapler staple "Game.app"
$ zip -r -q Game.dist.app Game.app

Please keep in mind that we can’t staple with Game.app.zip file.
Staple only apple to app bundle. When staples successfully, zip the Game.app again, then we can submit it to the user or SteamStore.

Submit to macOS App Store

Certificate Creation

Two different certificate are required when submit to macOS AppStore

  1. Mac App Distribution Certificate
  2. Mac Installer Distribution Certificate

Mac App Distribution Certificate is used for bundle CodeSign.

Mac Installer Distribution Certificate is used for pack bundle then uploading to itunes server for review.

Bundle Configuration

Same as Submit-to-SteamStore-and-non-AppStore-distribution

  1. delete all .meta file recursivly inside the Game.app folder.
  2. Create Game.entitlements file as informed.
  3. Modify Game.app/Contents/Info.plist if needed.
  4. Process third-party libs
  5. CodeSign all libs. (use Mac App Distribution)
  6. CodeSign Game.app. (use Mac App Distribution)

Generate .pkg file

After codesign, we need to pack Game.app into Game.app.pkg.

Use productbuild tool

1
2
3
4
productbuild --component ${BINARY} /Applications --sign "3rd Party Mac Developer Installer:" "${PKG_FILE}"

#sample
productbuild --component Game.app /Applications --sign "3rd Party Mac Developer Installer:" Game.app.pkg

Use AppLoader for uploading

Now we have Game.app.pkg bundle which is permitted for uploading.

It same process like iOS developement. use application loader tool for uploading to itunes server. Due to the removal of GUI version ApplcaitionLoader with Xcode11,
we can only submit with command line use xcrun altool.

  1. Verification before submission
1
xcrun altool --validate-app -f Game.app.pkg -u "${USER}" -p "${PWD}"

Tips: xcrun altool --validate-appwill not found all the erros, because apple itunes server will do the more stricter validation.

  1. Uploading
1
$ xcrun altool --upload-app -f Game.app.pkg -u "${USER}" -p "${PWD}"

If get some output like no error uploading package，means upload successfully.

Apple will also send an email when validation failed. Like this:
App Store Connect: Your app "XXXXX" (Apple ID: 123456789 Version: 1.0 Build: 0) has one or more issues

But if the validation got through, no email sent. Remember to check the itunes connect dashboard.

Fix all the errors that apple email mentions. After solve the errors, the process steps of codesign->pkg->upload must be executed completely. Do not forget to increse the build version number.
Every uploading task requires different build version number.

FAQ

I just list some common errors.

  • ITMS-90303: Unable to Sign

code object is not signed at all In subcomponent: PATH_TO_THE_FILE.
Delete these files before codesign.

  • ITMS-90238: Invalid Signature

Some libs are not signed, or signed with incorrect cert.

  • codesign throw error: CSSMERR_TP_CERT_REVOKED

The certificate is out-of-date, download the new one.

  • codesign 提示 errSecInternalComponent

If you do the codesign or productbuild via ssh, you should unlock the keychain with CLI. Because some process require user password to unlock the keychain. There is no password dialog pop up when these program request an access to the keychain.

Reference

Unity Manual: Delivering your application to the Mac App Store
gist:how-to-notarize-unity-for-macos.md
Notarizing macOS Software Before Distribution
Technical Note TN2206:macOS Code Signing In Depth

Thanks for reading, If there are any mistake or question, feel free to leave a comment below.