App crash on receiving notification using datastack

[imrobbyrc]

Crashlog: testflight_feedback.zip

the is intermittent, when the device recieved notifcation but apps is not running, it will crash. it doesnt happend when apps open or in background state.

my apps flow is, OnBoardingViewController is the main viewcontroller where apps open, its setup from datastack after that it'll setupTabBar like code above, here's some code for NotificationServiceExtension and other important code

hope you can help me with this @JohnEstropia

notification extension

class NotificationService: UNNotificationServiceExtension {

    var contentHandler: ((UNNotificationContent) -> Void)?
    var bestAttemptContent: UNMutableNotificationContent?

    override func didReceive(
        _ request: UNNotificationRequest,
        withContentHandler contentHandler: @escaping (UNNotificationContent) -> Void)
    {
        self.contentHandler = contentHandler
        bestAttemptContent = (request.content.mutableCopy() as? UNMutableNotificationContent)
        
        guard let bestAttemptContent = bestAttemptContent else {
            contentHandler(request.content)
            return
        }

        var data = bestAttemptContent.userInfo as! [String: Any]

        guard (try? notificationDataStack.perform(synchronous: { transaction in
            let notification = try transaction.importUniqueObject(Into<Notification>(),
                                                                  source: bestAttemptContent)
            data["id"] = notification?.id ?? ""
            bestAttemptContent.userInfo = data
            bestAttemptContent.badge = try transaction.fetchCount(From<Notification>(), Where<Notification>("read == %@", false)) as NSNumber
            return notification
        })) != nil else {
            contentHandler(bestAttemptContent)
            return
        }

        guard let attachment = request.attachment else {
            contentHandler(bestAttemptContent)
            return
        }
        bestAttemptContent.attachments = [attachment]
        contentHandler(bestAttemptContent)
    }
    
    override func serviceExtensionTimeWillExpire() {
        // Called just before the extension will be terminated by the system.
        // Use this as an opportunity to deliver your "best attempt" at modified content, otherwise the original push payload will be used.
        if let contentHandler = contentHandler, let bestAttemptContent =  bestAttemptContent {
            contentHandler(bestAttemptContent)
        }
    }

}

Datastack for notification extension

//
//  NotificationDataStack.swift
//  NotificationService
//
//  Created by Heryanto on 03/08/23.
//

import CoreStore

public var notificationDataStack: DataStack = {
    let dataStack = DataStack(
        CoreStoreSchema(
            modelVersion: "V1",
            entities: [
                Entity<Notification>("Notification"),
            ],
            versionLock: [
                "Notification": [0x22cf99de208e3ec2, 0x7829fbcec773a143, 0x2ce736cf3e20956b, 0x98417222b0f0cb7f]
            ]
        )
    )

    let fileURL = FileManager.default
        .containerURL(forSecurityApplicationGroupIdentifier: "group.marugame.udon")?
        .appendingPathComponent("MarugamePN.sqlite")

    try! dataStack.addStorageAndWait(
        SQLiteStore(
            fileURL: fileURL!,
            localStorageOptions: .allowSynchronousLightweightMigration
        )
    )
    return dataStack
}()

delegate for notification

extension SceneDelegate: UNUserNotificationCenterDelegate {
    func userNotificationCenter(
        _: UNUserNotificationCenter,
        willPresent _: UNNotification,
        withCompletionHandler completionHandler: @escaping (UNNotificationPresentationOptions) -> Void)
    {
        // Need to do this because observer doesn't notify by extension service
        try? notificationDataStack.perform(synchronous: { transaction in
            let notif = try! transaction.fetchOne(
                From<Notification>(),
                OrderBy<Notification>(.descending("created"))
            )
            notif?.read = false
        }, waitForAllObservers: true)
        completionHandler([.banner, .sound, .badge])
    }

my main datastack

class OnBoardingViewController: UIViewController {
   override func viewWillAppear(_ animated: Bool) {
       super.viewWillAppear(false)

       let dispatchGroup = DispatchGroup()

       dispatchGroup.enter()
       Stamps.createDataStack {
           dispatchGroup.leave()
       }

       dispatchGroup.enter()
       Omni.createDataStack {
           dispatchGroup.leave()
       }

       dispatchGroup.notify(queue: .main) {

           guard dataStack != nil || Omni.dataStack != nil else {
               print("Error: dataStack is not initialized")
               return
           }

           let storyboard = UIStoryboard(name: "Main", bundle: nil)
           let mainTabBarController = storyboard.instantiateViewController(identifier: "MarugameTabbarViewController")
           (UIApplication.shared.connectedScenes.first?.delegate as? SceneDelegate)?.changeRootViewController(mainTabBarController)
       }
   }
}

   func setupTabs(window: UIWindow?) {
       guard let window = window else { return }

       let homeNavigationController = HomeViewController.instantiate()
       let homeNavigationController2 = UINavigationController()
       homeNavigationController2.tabBarItem.title = "Scan QR"

       let orderNavigationController = UINavigationController(
           rootViewController: MarugameHostingController(rootView: OrderIndexView(), hidesNavigationBar: false, hidesBottomBar: false, navigationTitle: "Order History"))
       orderNavigationController.tabBarItem = UITabBarItem(title: "Orders",
                                                           image: UIImage(named: "tab-order-icon"), tag: 2)

       let accountNavigationController = UINavigationController(
           rootViewController: MarugameHostingController(rootView: AccountView(), hidesNavigationBar: false, hidesBottomBar: false, navigationTitle: "Account")
       )
       accountNavigationController.tabBarItem = UITabBarItem(title: "Account",
                                                             image: UIImage(named: "tab-account-icon"), tag: 3)

       let promotionNavigationController = UINavigationController(
           rootViewController: MarugameHostingController(rootView: PromotionView(), hidesNavigationBar: false, hidesBottomBar: false, navigationTitle: "Promotion"))
       promotionNavigationController.tabBarItem = UITabBarItem(title: "Promos",
                                                               image: UIImage(named: "tab-promotion-icon"), tag: 4)

       let viewControllers = [
           homeNavigationController,
           promotionNavigationController,
           homeNavigationController2,
           orderNavigationController,
           accountNavigationController
       ]

       if let tabBarController = window.rootViewController as? UITabBarController {
           tabBarController.viewControllers = viewControllers
           tabBarController.selectedIndex = TabBar.home.rawValue
       }
   }

   func changeRootViewController(_ vc: UIViewController, animated: Bool = true) {
       guard let window = window else { return }
       window.rootViewController = vc
       setupTabs(window: window)
       // add animation
       UIView.transition(with: window,
                         duration: 0.5,
                         options: [.transitionCrossDissolve],
                         animations: nil,
                         completion: nil)
   }
}

datastack

//
//  DataStack.swift
//  Marugame
//
//  Created by Robby Chandra on 21/06/23.
//

import Foundation
import CoreStore

public var dataStack: DataStack!

public enum Stamps {

    public static func createDataStack(onComplete: @escaping (() -> Void)) {
        dataStack = DataStack(
            CoreStoreSchema(
                modelVersion: "V1",
                entities: [
                    Entity<V1.Banner>("Banner"),
                    Entity<V1.Membership>("Membership"),
                    Entity<V1.Reward>("Reward"),
                    Entity<V1.RewardGroup>("RewardGroup"),
                    Entity<V1.Voucher>("Voucher"),
                    Entity<V1.VoucherGroup>("VoucherGroup"),
                    Entity<V1.Popup>("Popup"),
                ],
                versionLock: [
                    "Banner": [0x600e0c14f3174fb0, 0x4ad7883b64d50159, 0x61eb8f07e1a3ac42, 0xa9a8fdb774a082ba],
                    "Membership": [0x1dc90a60c450342a, 0xb67e77733adf96a8, 0x64763eba800d5273, 0x94b19686bf002575],
                    "Popup": [0x59e3bb649ca969c9, 0x67753f46d0fbafc1, 0x66c20abb7e408f53, 0x23c14ac6aa576899],
                    "Reward": [0x7b978dd5baf85d35, 0x8ca7bdb318cb025b, 0x7ec6fe07d986af88, 0xbb9b39bdbc340260],
                    "RewardGroup": [0xfcb44158673027ac, 0xd0ab5bcec8ad468c, 0x46bf8d5567c9cd02, 0x60a9c4ed47e6327c],
                    "Voucher": [0x19afba96c2fc110, 0xc0b3d8e36fa2f16b, 0xe58993540fde531e, 0x902a853074dbd3ed],
                    "VoucherGroup": [0x51f96443bfdd74b9, 0x157a1554c645e71a, 0x5e62338caf15efee, 0xf7a691441c6e36cf]
                ]
            ),
            CoreStoreSchema(
                modelVersion: "V2",
                entities: [
                    Entity<V2.Banner>("Banner"),
                    Entity<V2.Membership>("Membership"),
                    Entity<V2.Reward>("Reward"),
                    Entity<V2.RewardGroup>("RewardGroup"),
                    Entity<V2.Voucher>("Voucher"),
                    Entity<V2.VoucherGroup>("VoucherGroup"),
                    Entity<V2.Popup>("Popup"),
                ],
                versionLock: [
                    "Banner": [0x600e0c14f3174fb0, 0x4ad7883b64d50159, 0x61eb8f07e1a3ac42, 0xa9a8fdb774a082ba],
                    "Membership": [0x63b864e8cc8a2182, 0xad5f6fb0637c34b, 0xb736c06b81df2a9, 0x92d92d1c7086be6e],
                    "Popup": [0x59e3bb649ca969c9, 0x67753f46d0fbafc1, 0x66c20abb7e408f53, 0x23c14ac6aa576899],
                    "Reward": [0x7b978dd5baf85d35, 0x8ca7bdb318cb025b, 0x7ec6fe07d986af88, 0xbb9b39bdbc340260],
                    "RewardGroup": [0xfcb44158673027ac, 0xd0ab5bcec8ad468c, 0x46bf8d5567c9cd02, 0x60a9c4ed47e6327c],
                    "Voucher": [0x19afba96c2fc110, 0xc0b3d8e36fa2f16b, 0xe58993540fde531e, 0x902a853074dbd3ed],
                    "VoucherGroup": [0x51f96443bfdd74b9, 0x157a1554c645e71a, 0x5e62338caf15efee, 0xf7a691441c6e36cf]
                ]
            ),
            CoreStoreSchema(
                modelVersion: "V3",
                entities: [
                    Entity<Banner>("Banner"),
                    Entity<Membership>("Membership"),
                    Entity<Reward>("Reward"),
                    Entity<RewardGroup>("RewardGroup"),
                    Entity<Voucher>("Voucher"),
                    Entity<VoucherGroup>("VoucherGroup"),
                    Entity<Popup>("Popup"),
                ],
                versionLock: [
                    "Banner": [0x600e0c14f3174fb0, 0x4ad7883b64d50159, 0x61eb8f07e1a3ac42, 0xa9a8fdb774a082ba],
                    "Membership": [0x63b864e8cc8a2182, 0xad5f6fb0637c34b, 0xb736c06b81df2a9, 0x92d92d1c7086be6e],
                    "Popup": [0x59e3bb649ca969c9, 0x67753f46d0fbafc1, 0x66c20abb7e408f53, 0x23c14ac6aa576899],
                    "Reward": [0x7b978dd5baf85d35, 0x8ca7bdb318cb025b, 0x7ec6fe07d986af88, 0xbb9b39bdbc340260],
                    "RewardGroup": [0xfcb44158673027ac, 0xd0ab5bcec8ad468c, 0x46bf8d5567c9cd02, 0x60a9c4ed47e6327c],
                    "Voucher": [0x8a01201f48e41d4, 0xf69b81f62cd316c3, 0x21205dd2ef575eb9, 0xde79162edd1ad468],
                    "VoucherGroup": [0x51f96443bfdd74b9, 0x157a1554c645e71a, 0x5e62338caf15efee, 0xf7a691441c6e36cf]
                ]
            ),
            migrationChain: ["V1", "V2", "V3"]
        )

        try? dataStack.addStorage(SQLiteStore(
            fileName: "Marugame.sqlite",
            localStorageOptions: .allowSynchronousLightweightMigration
        )) { _ in
            onComplete()
        }
    }
}

public var previewDataStack: DataStack = {
    let dataStack = DataStack(
        CoreStoreSchema(
            modelVersion: "V2",
            entities: [
                Entity<Banner>("Banner"),
                Entity<Membership>("Membership"),
                Entity<Notification>("Notification"),
                Entity<Popup>("Popup"),
                Entity<Reward>("Reward"),
                Entity<RewardGroup>("RewardGroup"),
                Entity<Voucher>("Voucher"),
                Entity<VoucherGroup>("VoucherGroup"),
            ]
        )
    )
    try! dataStack.addStorageAndWait(InMemoryStore())
    return dataStack
}()

Homeviewcontroller where the crash is

class HomeViewController: UIViewController {

    @IBOutlet var guestHeader: UIView!
    @IBOutlet var memberHeader: UIView!
    @IBOutlet var memberName: UILabel!
    @IBOutlet var notificationIcon: UIImageView!
    @IBOutlet var memberPoints: UILabel!
    @IBOutlet var memberVouchers: UILabel!
    @IBOutlet var bannerSlider: ImageSliderView!
    @IBOutlet var activeOrderView: UIView!
    @IBOutlet var scrollView: UIScrollView!
    @IBOutlet var promotionCollectionView: PromotionCollectionView!
    @IBOutlet var activeOrderViewHeigtConstraint: NSLayoutConstraint!

    @IBOutlet var deliveryView: UIView!
    @IBOutlet var pickupView: UIView!
    @IBOutlet var dineInView: UIView!
    @IBOutlet var nearestStore: UIView!

    var refreshControl: UIRefreshControl?
    var membershipPublisher: ObjectPublisher<Membership>?
    let notificationPublisher: ListPublisher<Notification> = notificationDataStack.publishList(Notification.unreadClause)
    var voucherPublisher: ListPublisher<Voucher>?
    var bannerPublisher: ListPublisher<Banner>?
    let promotionPublisher: ListPublisher<Omni.Promotion> = Omni.dataStack.publishList(Omni.Promotion.forDisplayClause)
    let storePublisher: ListPublisher<Omni.Store> = StoreViewModel.storePublisher
    var userViewModel = UserViewModel()
    var subscribers = [AnyCancellable]()

    deinit {
        membershipPublisher?.removeObserver(self)
        notificationPublisher.removeObserver(self)
        voucherPublisher?.removeObserver(self)
        bannerPublisher?.removeObserver(self)
        Omni.Order.activeOrderPublisher.removeObserver(self)
        promotionPublisher.removeObserver(self)
        storePublisher.removeObserver(self)
    }

    class func instantiate() -> UINavigationController {
        let storyboard = UIStoryboard(name: "Home", bundle: nil)
        let controller = storyboard
            .instantiateViewController(
                withIdentifier: "HomeNavigationViewController") as! UINavigationController
        return controller
    }
    ```

[JohnEstropia]

Can you show how your DataStack is being initialized from UserViewModel.init()?
It's showing up in your stack trace:

[imrobbyrc]

here's my UserVIewModel.init()

class UserViewModel: ObservableObject {
    private let userPublisher = dataStack.publishList(From<Membership>().orderBy(.ascending(\.$id)))
    private let cartPublisher = Omni.dataStack.publishList(From<Omni.Cart>().orderBy(.ascending(\.$storeCode)))
    
    @ObjectState var user: ObjectSnapshot<Membership>?

    @Published var progress: Double = 0
    @Published var spendingToNextTier: Double = 0
    @Published var membershipStatus: MembershipStatus = .member

    init() {
        _user = .init(userPublisher.snapshot.first)
        progress = 0
        spendingToNextTier = 0
        userPublisher.addObserver(self, notifyInitial: true) { [weak self] publisher in
            guard let self else { return }
            let snapshot: ListSnapshot<Membership> = publisher.snapshot
            guard let user = snapshot.first else {
                self.progress = 0
                self.spendingToNextTier = 0
                self.membershipStatus = .member
                return
            }
            self._user = .init(user)
            let upgradeSpending = Double(user.upgradeSpending ?? 0)
            let totalSpending = Double(user.totalSpending ?? 0)
            let difference = upgradeSpending - totalSpending
            self.progress = upgradeSpending == 0 ? 100 : (upgradeSpending - difference) / upgradeSpending * 100
            self.spendingToNextTier = difference
            self.membershipStatus = MembershipStatus(rawValue: user.membershipStatus ?? 100) ?? .member
        }    }

    deinit {
        userPublisher.removeObserver(self)
    }
}

[imrobbyrc]

i got 3 variant of that crash

here's crashlytics trace
crashlytics.zip

[JohnEstropia]

The problem is this line:

Since your DataStack is initialized asynchronously, publishList() is trying to start a fetch before your addStorage()'s completion is called. I would recommend that you defer initializing any publishers on the DataStack before addStorage has completed. That said, I wouldn't recommend converting it to a synchronous version (addStorageAndWait()) either, as eventually you will have to handle migrations and convert them back to asynchronous again.

[imrobbyrc]

The problem is this line:

Since your DataStack is initialized asynchronously, publishList() is trying to start a fetch before your addStorage()'s completion is called. I would recommend that you defer initializing any publishers on the DataStack before addStorage has completed. That said, I wouldn't recommend converting it to a synchronous version (addStorageAndWait()) either, as eventually you will have to handle migrations and convert them back to asynchronous again.

hmm i think i already handle it in OnboardingViewController.

let dispatchGroup = DispatchGroup()

dispatchGroup.enter()
Stamps.createDataStack {
   dispatchGroup.leave()
}

dispatchGroup.enter()
Omni.createDataStack {
   dispatchGroup.leave()
}

dispatchGroup.notify(queue: .main) {

   guard dataStack != nil || Omni.dataStack != nil else {
       print("Error: dataStack is not initialized")
       return
   }

   let storyboard = UIStoryboard(name: "Main", bundle: nil)
   let mainTabBarController = storyboard.instantiateViewController(identifier: "MarugameTabbarViewController")
   (UIApplication.shared.connectedScenes.first?.delegate as? SceneDelegate)?.changeRootViewController(mainTabBarController)
}

[JohnEstropia]

I'm not sure when that OnboardingViewController code is getting called, but your crash is from HomeViewController → UserViewModel initialization. Try putting breakpoints and check whether the publishList calls are called ahead of your addStorage completion. Note that if you have an Storyboard set as an "Initial View Controller", that View Controller is likely to get initialized even before applicationDidFinishLaunching

[imrobbyrc]

I'm not sure when that OnboardingViewController code is getting called, but your crash is from HomeViewController → UserViewModel initialization. Try putting breakpoints and check whether the publishList calls are called ahead of your addStorage completion. Note that if you have an Storyboard set as an "Initial View Controller", that View Controller is likely to get initialized even before applicationDidFinishLaunching

try to breakpoint in publishList with print in appdelete and scene delete, it called after my print statement. do you have idea how i can handle addStorage ? i use OnboardingViewController to handle addStorage

[imrobbyrc]

hi @JohnEstropia , i got a reason why it crash, the initialize dataStack is complete but persistentStore is empty, you can see this log, do you know why it still empty? it's crash when the device in lock state

(lldb) po dataStack.txt

[JohnEstropia]

do you know why it still empty?

It's empty because addStorage() is not complete yet.

Add a breakpoint on the onComplete() line here:

Anything that accesses dataStack.* methods before that onComplete() runs is your culprit.

[imrobbyrc]

do you know why it still empty?

It's empty because addStorage() is not complete yet.

Add a breakpoint on the onComplete() line here:

Anything that accesses dataStack.* methods before that onComplete() runs is your culprit.

thanks for reply @JohnEstropia, but i use DispatchGroup() to handle the flow

        let dispatchGroup = DispatchGroup()

        dispatchGroup.enter()
        Omni.createDataStack {
            dispatchGroup.leave()
        }

        dispatchGroup.enter()
        Stamps.createDataStack {
            dispatchGroup.leave()
        }

        dispatchGroup.notify(queue: .main) {
            let storyboard = UIStoryboard(name: "Main", bundle: nil)
            let mainTabBarController = storyboard.instantiateViewController(identifier: "MarugameTabbarViewController")
            (UIApplication.shared.connectedScenes.first?.delegate as? SceneDelegate)?.changeRootViewController(mainTabBarController)
            if Membership.isAuthenticated, !Defaults.deviceToken.isEmpty {
                registerPushNotificationKey()
            }
            Session.setCrashlyticsData()
            UIApplication.shared.registerForRemoteNotifications()
        }
    }

onComplete() metho should call dispatchGroup.leave() and will notify dispatchGroup, right?

[imrobbyrc]

i got this error on corestore callback @JohnEstropia

[JohnEstropia]

Sorry if I wasn't clear. The breakpoints are to find the order of events happening in your app. You'd need to compare which of the following is hit first:

• addStorage() method's onComplete() callback
• HomeViewController.promotionPublisher let stored property (I'm assuming Omni.dataStack is the same as the global var dataStack declared in DataStack.swift)
• UserViewModel.init(), specifically let userPublisher and let cartPublisher
• Anything else that is calling methods on the dataStack that happens right after launch

[JohnEstropia]

Sorry, scratch that. I just noticed your error in the last screenshot is in the completion closure itself. The error code 259 implies the file already exists. Do you know where/when this file is being created?

[imrobbyrc]

Sorry, scratch that. I just noticed your error in the last screenshot is in the completion closure itself. The error code 259 implies the file already exists. Do you know where/when this file is being created?

the file should be exists because if i open the apps normaly it will be fine, the problem is when the apps backgroundly launched by push notification in locked state

[JohnEstropia]

I see. CoreStore doesn't set any NSFileProtectionKey or NSPersistentStoreFileProtectionKey explicitly, so the default will be NSFileProtectionCompleteUntilFirstUserAuthentication. Does the same issue happen if the notifications arrive in the lockscreen AFTER the first device passcode entry?

[imrobbyrc]

I see. CoreStore doesn't set any NSFileProtectionKey or NSPersistentStoreFileProtectionKey explicitly, so the default will be NSFileProtectionCompleteUntilFirstUserAuthentication. Does the same issue happen if the notifications arrive in the lockscreen AFTER the first device passcode entry?

yes, as long as the device is locked it will be fail to initiate datastack

[imrobbyrc]

how i can set data NSPersistentStoreFileProtectionKey and NSFileProtectionKey ? i see in source code it only let variable

[JohnEstropia]

I'll add a way to set these flags. In the meantime, if you need it sooner you might want to fork CoreStore temporarily and add the necessary flags in the code you screenshot above

[imrobbyrc]

I'll add a way to set these flags. In the meantime, if you need it sooner you might want to fork CoreStore temporarily and add the necessary flags in the code you screenshot above

yes i need it sooner as my crashlytics going down 😅 thankyou for your help @JohnEstropia

[JohnEstropia]

Another approach would be to present a separate notification when the device has not been authenticated. If your users' data are loaded from that store, you might want to keep them encrypted for security.

Regardless, I'll notify this thread after I update the SDK

[imrobbyrc]

Another approach would be to present a separate notification when the device has not been authenticated. If your users' data are loaded from that store, you might want to keep them encrypted for security.

Regardless, I'll notify this thread after I update the SDK

i think i will wait for SDK. thanks @JohnEstropia

[imrobbyrc]

hi sorry, any update on this bro @JohnEstropia ?

[JohnEstropia]

@imrobbyrc Since I cannot promise a release date on this update, please fork and implement the protection flags on your side for now. While this looks like an easy fix for some simple use cases, supporting it on the API level means CoreStore should be able to handle any failures related to access protections so I'd like to do it correctly for common use cases (ex: deeplinks from push notifications like yours, background fetches etc.)