Shulou(Shulou.com)06/02 Report--

This article mainly explains the reason analysis of using SQLite and FMDB instead of Core Data. The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn the reason analysis of using SQLite and FMDB instead of Core Data.

Why don't I use Core Data?

Mike Ash wrote: as far as I'm concerned, I'm not a big fan. I found that API is clumsy, and the framework itself is extremely slow for large amounts of data.

A practical example: 10000 entries

Imagine a RSS reader where a user can right-click on a feed and select mark all as read.

Under the engine, there is an Article entity with the read attribute. To mark all entries as read, the program needs to load all articles for this feed (possibly through an one-to-many relationship), and then set the read property to YES.

For the most part, it doesn't matter. But imagine that there are 200 articles in that feed, and to avoid blocking the main thread, you might consider doing this in the background thread (especially if your program is an iPhone application). When you start using Core Data multithreading, things start to get harder to handle.

This may be all right, at least it's not worth switching to Core Data.

But then add synchronization.

I have used two different RSS synchronization interfaces to get the ID list of read articles. One of them returns nearly 10000 ID.

You don't plan to load 10000 articles in the main thread and then set read to NO. You don't even want to load 10000 articles in a background thread, even if you manage memory carefully, which is too much work (if you do this frequently, think about the impact on battery life).

What you really want to do is have the database set read to YES for every article in the ID list.

SQLite can do this with a single call. Assuming that there is an index on uniqueID, this will be very fast. And you can execute in the background thread as easily as in the main thread.

Another example: quick start

I want to reduce the startup time of my other program, not just the start time, but all the time before the data is displayed.

That's a Twitter-like application (although it's not) that displays the timeline of the message. Displaying the timeline means getting messages and loading relevant users. It's fast, but when it starts, it populates the UI and then the data.

My theory about iPhone applications (or all applications) is that startup time is more important than most other developers think. The app starts slowly and doesn't look like it's going to start, because people subconsciously remember it and come up with the idea of preventing it from starting. Reducing startup time reduces friction, making users more likely to continue to use your application and recommend it to others. This is part of how you make your application work.

Because I don't use Core Data, I have a simple, conservative solution on hand. I save timeline (messages and people objects) to a plist file via NSCoding. When it starts, it reads the file, creates messages and character objects, and displays the timeline as soon as the UI appears.

This significantly reduces latency.

It is impossible to use message and character objects as instance objects of NSManagedObject. Suppose I have encoded and stored IDs objects, but that means reading plist and then touching the database. I completely avoided the database in this way.

I removed the code after the newer and faster machine came out. Looking back, I wish I could keep it.

How do I think about this?

When considering whether to use Core Data, I consider the following things:

Will there be an incredible amount of data?

For a RSS reader or Twitter app, the answer is obvious: yes. Some people follow hundreds of people. A person may subscribe to thousands of feeds.

Even if your application does not get data from the network, it is still possible for users to add data automatically. If you use a Mac that supports AppleScript, some people will write scripts to load a lot of data. It's the same if you add data through web API.

Will there be a Web API containing a terminal similar to a database (compared to a class object terminal)?

An RSS synchronization API can return a uniquelIDs list of articles that have been read. A synchronous API of a note-taking application may return the uniquelIDs of archived and deleted notes.

Is it possible for the user to handle a large number of objects through actions?

At the bottom, you need to consider the same problem as before. When someone deletes all 5000 downloaded pasta recipes, how well can your recipe app do this (on iPhone? )?

When I decide to use Core Data (I have released an application that uses Core Data), I will pay careful attention to how I use it. For good performance, I found that I used it as a strange interface to a SQL database, and then I knew I should abandon Core Data and use SQLite directly.

How do I use SQLite

I use SQLite,FMDB through FMDB Wrapper from Flying Meat Software, provided by Gus Mueller.

Basic operation

I used SQLite before iPhone, and Core Data used it before. This is the main point of how it works:

All database access-read and write-occurs in a continuous queue, in a background thread. Touching the database in the main thread is never allowed. Use a continuous queue to ensure that everything happens in order.

I use blocks a lot to make asynchronous programs easier.

The model object exists only in the main thread (with two important exceptions), and the change triggers a background save.

Model objects list the properties they store in the database. It could be in the code or in the plist file.

Some model objects are unique and some are not. It depends on the needs of the application (in most cases it is unique).

For relational data, I try to avoid linked table queries as much as possible.

Some object types are fully read into memory at startup, while others may only need to create and maintain one of their uniqueIDs's. NSMutableSet, so I don't need to touch the database to know what's already there.

Calls to Web API occur in background threads, which use separate model objects.

I will describe it in detail through the code of my current application.

Database update

In my recent application, there is a single database controller, VSDatabaseController, which talks to SQLite through FMDB.

FMDB distinguishes between updates and queries. To update the database, call app:

-[VSDatabaseController runDatabaseBlockInTransaction: (VSDatabaseUpdateBlock) databaseBlock]

VSDatabaseUpdateBlock is simple:

Typedef void (^ VSDatabaseUpdateBlock) (FMDatabase * database)

RunDatabaseBlockInTransaction is also simple:

-(void) runDatabaseBlockInTransaction: (VSDatabaseUpdateBlock) databaseBlock {dispatch_async (self.serialDispatchQueue, ^ {@ autoreleasepool {[self beginTransaction]; databaseBlock (self.database); [self endTransaction];}});}

Notice that I use my own continuous scheduling queue. Gus recommends taking a look at FMDatabaseQueue, which is also a continuous scheduling queue. I haven't been able to take a look at it because it's newer than everything else in FMDB. )

The calls to beginTransaction and endTransaction are nested (in my database controller). They will call-[FMDatabase beginTransaction] and-[FMDatabase commit] when appropriate. Using transactions is the key to making SQLite faster. Tip: I store the current transaction in-[NSThread threadDictionary]. It's easy to get data from each thread, and I almost never use anything else.

Here is a simple example of calling to update the database:

-(void) emptyTagsLookupTableForNote: (VSNote *) note {NSString * uniqueID = note.uniqueID; [self runDatabaseBlockInTransaction: ^ (FMDatabase * database) {[database executeUpdate: @ "delete from tagsNotesLookup where noteUniqueID =?;", uniqueID];};}

That means something. First of all, SQL is not scary. Even if you've never seen it, you know what this line of code does.

Like all other public interfaces of VSDatabaseController, emptyTagsLookupTableForNote should be called in the main thread. Model objects can only be referenced in the main thread, so uniqueID is used in block instead of VSNote objects.

Note that in this case, I updated a lookup table. Notes and tags are many-to-many relationships, and one way to express them is to map note uniqueIDs and tag uniqueIDs with a database table. These tables won't be difficult to maintain, but I do try to avoid their use if possible.

Notice the? in the update string. -[FMDatabase executeUpdate:] is a variable argument function. SQLite supports the use of placeholders, so you don't need to put authentic values in strings. Here's a security problem: it helps daemons object to SQL inserts. If you need to avoid certain values, it also saves you trouble.

*, in the tagsNotesLookup table, there is an index for noteUniquelID (the index is another key to SQLite performance). This line of code is called every time it starts:

[self.database executeUpdate: @ "CREATE INDEX if not exists noteUniqueIDIndex on tagsNotesLookup (noteUniqueID);"]

Database acquisition

To get the object, app calls:

-[VSDatabaseController runFetchForClass: (Class) databaseObjectClass fetchBlock: (VSDatabaseFetchBlock) fetchBlock fetchResultsBlock: (VSDatabaseFetchResultsBlock) fetchResultsBlock]

These two lines of code do most of the work:

FMResultSet * resultSet = fetchBlock (self.database); NSArray * fetchedObjects = [self databaseObjectsWithResultSet:resultSet class:databaseObjectClass]

Use FMDB to find the database and return a FMResultSet. With resultSet you can loop through sentence by sentence to create model objects.

I suggest writing generic code to convert database rows to objects. One method I use is to map column names to object properties with a plist. It also contains types, so you know if you need to call-[FMResultSet dateForColumn:],-[FMResultSet stringForColumn:] or something else.

I did some simple things in my * * app. The database row happens to correspond to the name of the model object property. All properties are strings, except for those whose names end with "Date". It's simple, but you can see that a clear correspondence is needed.

Unique object

Create a model object and get data from the database in the same background thread. As soon as it gets it, the program transfers them to the main thread.

Usually I have uniqued objects. The same database row result always corresponds to the same object.

To be unique, I created an object cache, a NSMapTable, in the init function: _ objectCache = [NSMapTable weakToWeakObjectsMapTable]. Let me explain:

For example, when you make a database fetch and transfer objects to a view controller, you want them to disappear after the view controller has finished using them, or when a different view controller displays them.

If your object cache is a NSMutableDictionary, you will need to do some extra work to clear the objects in the cache. It becomes painful to determine whether its corresponding object is referenced somewhere else. If NSMapTable is a weak reference, it will deal with this problem automatically.

So: we make the object unique in the main thread. If an object already exists in the object cache, we use that existing object. The main thread wins because it may have new changes. If it is not in the object cache, it will be added

Keep objects in memory

Many times, it makes sense to keep the entire object type in memory. My app has a VSTag object. Although there may be hundreds or thousands of notes, the number of tags is very small, basically less than 10. A tag has only six attributes: three BOOL, two small NSstring, and one NSDate.

On startup, app takes all the tags and saves them in two dictionaries, one primary key is tag's uniqueID, and the other is the lowercase tag name.

This simplifies a lot of things, not just the tag auto-completion system, which can be operated completely in memory without the need for database access.

But many times, it is impractical to keep all the data in memory. For example, we don't keep all our notes in memory.

But there are times when you can't keep objects in memory, you want to keep all the uniqueIDs in memory. You will do a get like this:

FMResultSet * resultSet = [self.database executeQuery:@ "select uniqueID from some_table"]

ResultSet contains only uniqueIDs, which you can store in a NSMutableSet.

I find this useful for web APIs sometimes. Imagine an API call that returns a list of uniqueIDs notes that have been created since a certain time. If I already have a local NSMutableSet containing all the notes uniqueIDs, I can quickly check (via-[NSMutableSet minusSet]) to see if there are any missing notes, and then call another API to download those missing notes. These do not need to touch the database at all.

However, things like this should be handled with care. Can app provide enough memory? Does it really simplify programming and improve performance?

Using SQLite and FMDB instead of Core Data gives you a lot of flexibility and room for smart solutions. Remember that sometimes being smart is good, and sometimes being smart is a big mistake.

Web APIs

My API call is in the background process (I often use a NSOperationQueue, so I can cancel the operation). The model object is only in the main thread, but I also pass the model object to my API call.

Here's the thing: a database object has a detachedCopy method that replicates database objects. This replicated object is not an object cache used by the reference itself to unify. The only place to reference that object is the API call, and when the API call ends, the copied object disappears.

This is a good system because it means that I can use model objects in API calls. The method looks like this:

-(void) uploadNote: (VSNote *) note {VSNoteAPICall * apiCall = [[VSNoteAPICall alloc] initWithNote: [note detachedCopy]]; [self enqueueAPICall:apiCall];}

VSNoteAPICall takes the value from the copied VSNote and creates a HTTP request instead of a dictionary or other form of note representation.

Processing Web API return value

I did something similar with the web return value. I will create a model object for the returned JSON or XML, which is also separate. It is not stored in a model cache for uniqueness.

There are some things here that are uncertain. Sometimes it is necessary to use that model object to make local modifications in two places: the in-memory cache and the database.

The database is usually the easy part. For example, my application already has a way to save note objects. It uses a SQL insert or replace string. I just need to call the note object generated by the return value from web API, and the database will be updated.

But maybe there is a version of that object in memory, and fortunately we can easily find it:

VSNote * cachedNote = [self.mapTable objectForKey:downloadedNote.uniqueID]

If cachedNote exists, I'll let it get a value from downloadedNote instead of replacing it (which may violate uniqueness). This can share the code of the detachedCopy method.

Once the cachedNote is updated, the viewer will notify the note via KVO, or I will send a NSNotification, or both.

The Web API call also returns some other values. I mentioned that RSS readers might get a large list of read items. In this case, I create a NSSet with that list, update the read property of each cached article in memory, and then call-[FMDatabase executeUpdate:].

The key to making it work fast is that NSMapTable lookups are fast. If you are looking for someone in a NSArray, we should reconsider.

Database migration

Core Data's database migration is cool when it works.

But inevitably, it is a layer of code and database. The more directly you use SQLite, the more directly you update the database.

You can do this safely and easily.

For example, add a table:

[self.database executeUpdate:@ "CREATE TABLE if not exists tags" (uniqueID TEXT UNIQUE, name TEXT, deleted INTEGER, deletedModificationDate DATE); "]

Or add an index:

[self.database executeUpdate:@ "CREATE INDEX if not exists"archivedSortDateIndex on notes (archived, sortDate);"]

Or add a column:

[self.database executeUpdate:@ "ALTER TABLE tags ADD deletedDate DATE"]

The application should use the above code to set up the database in * * places of the code. For future changes, just add a call to executeUpdate-I told them to do it sequentially. Because I designed my database, there won't be any problems (I've never had a performance problem, it's fast).

Of course, big changes require more code. If your data is obtained through web, sometimes you can start with a new database model and re-download the data you need.

Performance skills

SQLite can be very fast, or it can be very slow. It all depends on how you use it.

Business

Wrap the update in a transaction. Call-[FMDatabase beginTransaction] before the update and-[FMDatabase commit] after the update.

If you have to Denormalize

Anti-standardization is very unpleasant. This approach is to add redundant data to speed up retrieval, but it means that you need to maintain redundant data.

I always avoid it crazily until there is a serious performance difference. Then I will do it as little as possible.

Use index

The create statement for the tags table in my application looks like this:

CREATE TABLE if not exists tags (uniqueID TEXT UNIQUE, name TEXT, deleted INTEGER, deletedModificationDate DATE)

The uniqueID column is automatically indexed because it is defined as unique. But if I wanted to query the table with name, I might create an index on name, like this:

CREATE INDEX if not exists tagNameIndex on tags (name)

You can create indexes on multiple columns at once, like this:

CREATE INDEX if not exists archivedSortDateIndex on notes (archived, sortDate)

But note that too many indexes will slow down your insertion speed. All you need is enough and the right ones.

Use the command line application

When my app runs in the simulator, I print the path to the database. I can open the database from the command line of sqlite3. Learn more about this application through the man sqlite3 command.

The command to open the database: sqlite3 "path to the database".

After opening it, you can look at schema: type. Schema.

You can update and query, which is a good way to check that the SQL is correct before using your app.

The coolest part of this is the SQLite Explain Query Plan command, and you'll want to make sure your statements are executed as quickly as possible.

A real example

My app displays a list of all the tags that don't have archived notes. Whenever there is a change in notes or tags, the query is re-executed, so it needs to be fast.

I can use SQL join to query, but it's slow (joins is slow).

So I gave up sqlite3 and started trying something else. I looked at my schema again and realized that I could de-normalize. The archived status of a note can be stored in the notes table, or it can be stored in the tagsNotesLookup table.

Then I can execute a query:

Select distinct tagUniqueID from tagsNotesLookup where archived=0

I already have an index on tagUniqueID. So I use explain query plan to tell me what happens when I execute this query.

Sqlite > explain query plan select distinct tagUniqueID from tagsNotesLookup where archived=0; 0 | 0 | 0 | SCAN TABLE tagsNotesLookup USING INDEX tagUniqueIDIndex (~ 100000 rows)

It uses an index, but SCAN TABLE doesn't sound good. * * is a SEARCH TABLE and overrides an index.

I indexed tagUniqueID and archive:

CREATE INDEX archivedTagUniqueID on tagsNotesLookup (archived, tagUniqueID)

Execute explain query plan again:

Sqlite > explain query plan select distinct tagUniqueID from tagsNotesLookup where archived=0; 0 | 0 | 0 | SEARCH TABLE tagsNotesLookup USING COVERING INDEX archivedTagUniqueID (archived=?) (~ 10 rows)

Much better.

More performance tips

The ability to cache statements is added somewhere in FMDB, so when I create or open a database, I always call [self.database setShouldCacheStatements:YES]. This means that you do not need to compile each statement again for each call.

I have never found a good guide to using vacuum, and if the database is not compressed regularly, it will get slower and slower. My app runs a vacuum, but only once a week (it stores the time of the last vacuum in NSUserDefaults, and then checks to see if a week has passed at the beginning).

If you can auto_vacuum, that's even better. Look at the pragma statements supported by SQLite list.

Other cool things.

Gus Mueller asked me to cover the contents of custom SQLite methods. I didn't really use these things, and now that he pointed it out, I can rest assured that I can find its use. Because it's cool.

In Gus's post, there is a query that looks like this:

Select displayName, key from items where UTTypeConformsTo (uti,?) Order by 2

SQLite knows nothing about UITypes. But you can add the core method and see-[FMDatabase makeFunctionNamed:maximumArguments:withBlock:].

You can execute a large query to replace it, and then evaluate each object. But that takes more work. * * filter at the SQL level, not after the table row is converted to an object.

Thank you for your reading. the above is the content of "reason Analysis with SQLite and FMDB instead of Core Data". After the study of this article, I believe you have a deeper understanding of the reason analysis of using SQLite and FMDB instead of Core Data, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.