Shulou(Shulou.com)05/31 Report--

In this article, the editor introduces "GIT LFS migration case analysis" in detail, with detailed content, clear steps and proper handling of details. I hope this "GIT LFS migration case analysis" article can help you solve your doubts.

GIT LFS (Large File Storage)

Although git has never been suitable for managing binaries, git now seems to provide git lfs, a plug-in dedicated to managing large files, by default.

The basic principle is simply to use a similar file pointer (text) instead of the actual file storage. Git only stores the change history of the file pointer rather than the entire binary file, and automatically provides hook when it is used, so it is convenient for operations such as clone, pull, reset and so on to automatically obtain the source binaries of these file pointers and update the binary file commit as well. Git will automatically transfer the source file to the file pointer to git log, while the source file will be uploaded to lfs. So at the user level, the use of GIT LFS is actually insensitive.

Transfer

The above gives a brief introduction to GIT LFS, and then goes straight to how to migrate, as to why we talk about migration instead of how to use LFS from 0.

Because it is often the git warehouse used to find that the warehouse is so large, clone is so slow, and then think about using LFS.

Migration requires that we have administrator privileges for the warehouse and will unprotect branches and so on.

The specific LFS migration is mainly divided into the following steps.

It is best to make a backup before migration and communicate well with your team colleagues. After all, the operation involves-f high-risk operations and is easy to carry the pot.

GIT LFS server configuration

For some self-built git services, you may need to configure the server to enable LFS, such as gitlab.

GIT LFS client installation

Windows's git installation package comes with this plug-in, which does not need to be installed separately. Other platforms can install it on their own, link.

Try the following command on the command line.

Git lfs

If you have a similar output of help document information, you already have a git lfs client.

Git-lfs/2.11.0 (GitHub; windows amd64; go 1.14.2; git 48b28d97) git lfs [] Git LFS is a system for managing and versioning large files inassociation with a Git repository. Instead of storing the large fileswithin the Git repository as blobs, Git LFS stores special "pointerfiles" in the repository, while storing the actual file contents on aGit LFS server. The contents of the large file are downloadedautomatically when needed, for example when a Git branch containingthe large file is checked out.Git LFS works by using a "smudge" filter to look up the large filecontents based on the pointer file, and a "clean" filter to create anew version of the pointer file when the large file's contents change.It also uses a pre-push hook to upload the large filecontents tothe Git LFS server whenever a commit containing anew large fileversion is about to be pushed tothe corresponding Git server.

Then you need to execute the following command to configure the LFS global environment, which only needs to be configured once. At the same time, the hooks of the current warehouse will be updated.

Git lfs install

Migrate the local history warehouse

The basic idea of lfs migration: lfs rewrites local history-> force push overwrites the remote end to achieve the effect of migration.

So we'd better synchronize the local repository with the remote and create a local branch for all the remote branches.

Then cd goes to his local warehouse and executes the following command. There is a glob expression in-include. Add the file name you want to manage by LFS.-everything represents all local branches.

Git lfs migrate import-- include= "* .bin,*.lib,*.so,*.dll,*.a,*.param,*.zip,*.gz"-- everythingmigrate: Sorting commits:..., done.migrate: Rewriting commits: 100,100,193), done. Develop bacb490a80ea46d73bd3866c2e7cf7ad199ce5eb-> 72884bcb4629417bad73ea3d485d08a0708909cd feature/npu-platform a3645632756becc527c7f4d58514b3c479f824d3-> e227900a3903b3a6955e4dffee48daeceac6cdff master 1ccdecdcb4b5d6224a6e24c6f87793bfcc15ee4c-> 1d9fc2139600ef3d92a20d65bb5db89021b8c488 0.1.0 07c6b2aa732506f1cc88cedb551f37f376b6efa6-> 8e55193221dfca9f6bb28ccd9cca85af9c5958c9 1.0.0 0f694efcd7aa9df641836e1ea6eebbb730b940b5-> 3f9e77575120b6e56b34790c998a362116da75f5migrate: Updating refs:..., done.

Rewrite the local branch, tag and so on.

Here, we can first execute git lfs ls-files to see which files have been transferred to lfs management and check if there are any omissions.

At this time, no matter in which branch, the .gitattributes file will appear and will be added something similar to the following.

* .bin filter=lfs diff=lfs merge=lfs-text*.lib filter=lfs diff=lfs merge=lfs-text*.so filter=lfs diff=lfs merge=lfs-text*.dll filter=lfs diff=lfs merge=lfs-text*.a filter=lfs diff=lfs merge=lfs-text*.param filter=lfs diff=lfs merge=lfs-text*.zip filter=lfs diff=lfs merge=lfs-text*.gz filter=lfs diff=lfs merge=lfs-text

At the same time, you can see that all of our binaries have been converted to the following form of text

Version https://git-lfs.github.com/spec/v1oid sha256:9171c8350d72ccca6ad60ac80b577157ad1f9fd44ca05744216e02ccbfcdf491size 10260

Confirm it is correct, and then you can push it to the remote end.

Since the migration of lfs rewrites all commit and modifies the hash value, we need to add-froce

This step requires unprotecting the branch (the protected branch cannot be-f)

Git push-force-all

In this way, the lfs migration of the remote warehouse is completed

Some supplementary instructions for migration

The lfs file of the local repository of the migrant is transferred to the source file: after the above steps, since all the files have been transferred to file pointers, we need to download the files back to use the repository normally.

It should be noted that this step is not required for others to re-clone or synchronize lfs migrated remote repositories, only for migrating the author's local repository.

Git lfs pull

Synchronization of the local repository before the migration of other members of the team: since the history of the remote repository has been rewritten, it is impossible to synchronize directly. It is best to delete the local branch and pull the remote branch again. If the local commit already needs to be submitted, you can rename the local branch and pull the remote to do cherry pick. Git tag similarly, delete the tag before migration.

Local warehouse cleanup: the above migration successfully migrates binaries into git lfs objects, and git log is not storing source file changes but pointer changes, but there is still a git log cache in the local .git folder that is no longer needed. Execute the following command to clean up.

Git reflog expire--expire-unreachable=now-- allgit gc-- prune=now Warehouse comparison before and after cleaning

Intuitively speaking, lfs is more aimed at the slow clone of large warehouses. Before and after lfs migration, we each back up a small remote warehouse for testing.

The test warehouse binaries used are relatively small, with a total size of 50m, and the number of changes is also in single digits.

Clone down the warehouse size comparison.

Similar to my estimate, generally speaking, it is more suitable for binary files to change frequently. If the file is large, but the file does not change, there is little difference in clone. After all, lfs still has steps to download source files in clone. Apart from downloading, operating files means that there will still be theoretical performance improvement for git, but the perception may not be strong.

After reading this, the article "case Analysis of GIT LFS Migration" has been introduced. If you want to master the knowledge points of this article, you still need to practice and use it yourself. If you want to know more about related articles, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.