Monorepos, Submodules & Migration
You are entering advanced enterprise repository management where you'll architect and manage massive codebases that span multiple teams, projects, and technologies across global organizations.
Commander, managing version control for enterprise organizations is like coordinating multiple space stations, each with dozens of modules, hundreds of crew members, and thousands of interconnected systems. Just as the International Space Station requires sophisticated coordination between different modules from various countries, enterprise software development requires advanced strategies for managing massive repositories that contain multiple projects, shared libraries, and complex dependencies.
You'll master Large Repository Management and Monorepo Architecture - the advanced organizational strategies that enable teams to scale version control across enterprise environments. From Git submodules to monorepo patterns, from repository splitting to performance optimization, you'll build the expertise needed to architect version control systems that support thousands of developers working on interconnected projects.
The fundamental architectural decision for enterprise version control is choosing between monorepo (single repository) and polyrepo (multiple repositories) strategies. Each approach has distinct advantages and challenges that must align with your organization's structure, team dynamics, and technical requirements.
A single repository containing multiple projects, shared libraries, and related codebases managed together.
Multiple independent repositories, each containing a specific project or service with clear boundaries.
| Criteria | Monorepo | Polyrepo | Hybrid |
|---|---|---|---|
| Team Size | Small to Medium Teams | Large Distributed Teams | Medium to Large Teams |
| Code Sharing | Frequent Cross-Project | Limited Sharing | Selective Sharing |
| Deployment Frequency | Coordinated Releases | Independent Releases | Mixed Release Cycles |
| Technology Stack | Uniform Stack | Diverse Technologies | Mixed Technologies |
| Security Requirements | Uniform Access | Granular Control | Selective Control |
Scale: 2+ billion lines of code, 25,000+ developers
Tools: Piper (internal), Bazel build system
Benefits: Atomic changes, shared infrastructure, unified standards
Scale: Thousands of repositories, microservices architecture
Strategy: Service ownership, independent deployments
Benefits: Team autonomy, technology diversity, clear boundaries
Approach: Monorepo for core platforms, polyrepo for products
Tools: Git Virtual File System (VFS for Git)
Benefits: Flexibility, selective sharing, performance optimization
Git submodules enable you to include external repositories as subdirectories within your main repository while maintaining independent version control. This is essential for managing shared libraries, third-party dependencies, and modular architectures in enterprise environments.
# Add a submodule to your repository
git submodule add https://github.com/company/shared-components.git lib/shared-components
# Add submodule to specific directory with custom name
git submodule add https://github.com/company/ui-toolkit.git frontend/components/ui-toolkit
# Add submodule from specific branch
git submodule add -b develop https://github.com/company/api-client.git lib/api-client
# Commit the submodule addition
git add .gitmodules lib/shared-components
git commit -m "feat: add shared-components submodule for reusable UI elements"# Clone repository and initialize all submodules
git clone --recurse-submodules https://github.com/company/main-project.git
# Alternative: clone first, then initialize submodules
git clone https://github.com/company/main-project.git
cd main-project
git submodule init
git submodule update
# Initialize and update in one command
git submodule update --init --recursive
# Pull latest changes in all submodules
git submodule update --remote --recursive# Update specific submodule to latest commit
cd lib/shared-components
git pull origin main
cd ../..
git add lib/shared-components
git commit -m "chore: update shared-components to latest version"
# Update all submodules to latest remote commits
git submodule update --remote
# Update submodules and automatically merge changes
git submodule update --remote --merge
# Update submodules and automatically rebase local changes
git submodule update --remote --rebase
# Check submodule status
git submodule status
git submodule summaryMultiple projects include the same shared library as a submodule, ensuring consistent versions across applications.
# Project A
lib/
├── auth-service/ (submodule)
├── logging-utils/ (submodule)
└── ui-components/ (submodule)
# Project B
dependencies/
├── auth-service/ (same submodule)
├── logging-utils/ (same submodule)
└── payment-gateway/ (submodule)Submodules containing their own submodules, creating hierarchical dependency structures for complex enterprise architectures.
# Main Project
├── frontend/ (submodule)
│ ├── components/ (nested submodule)
│ └── themes/ (nested submodule)
├── backend/ (submodule)
│ ├── auth/ (nested submodule)
│ └── database/ (nested submodule)Always pin submodules to specific commits or tags, not branch heads, for reproducible builds.
Establish cadence for submodule updates with proper testing and validation processes.
Ensure team members have appropriate access to all submodule repositories.
Automate submodule updates in CI/CD pipelines with dependency scanning.
Git subtrees provide an alternative to submodules by directly incorporating external repositories into your project's history. This approach eliminates many submodule complexities while maintaining the ability to synchronize with upstream repositories.
| Feature | Git Subtrees | Git Submodules |
|---|---|---|
| Repository Integration | Fully integrated, part of main repo | Referenced, separate repositories |
| Cloning Complexity | Standard git clone works | Requires --recurse-submodules |
| History Tracking | Squashed or merged history | Preserves separate history |
| Upstream Contributions | More complex push process | Direct contribution workflow |
| Repository Size | Increases main repo size | Keeps repos separate |
| Team Onboarding | No special knowledge needed | Requires submodule understanding |
# Add remote repository as subtree
git subtree add --prefix=lib/shared-utils https://github.com/company/shared-utils.git main --squash
# Add subtree from specific branch or tag
git subtree add --prefix=vendor/third-party https://github.com/vendor/library.git v2.1.0 --squash
# Add subtree without squashing history
git subtree add --prefix=modules/auth https://github.com/company/auth-service.git main# Pull latest changes from upstream
git subtree pull --prefix=lib/shared-utils https://github.com/company/shared-utils.git main --squash
# Pull specific version
git subtree pull --prefix=vendor/third-party https://github.com/vendor/library.git v2.2.0 --squash
# Strategy for regular updates with remotes
git remote add shared-utils-remote https://github.com/company/shared-utils.git
git subtree pull --prefix=lib/shared-utils shared-utils-remote main --squash# Push changes made in subtree back to upstream
git subtree push --prefix=lib/shared-utils https://github.com/company/shared-utils.git feature-branch
# Push to remote with branch creation
git subtree push --prefix=lib/shared-utils shared-utils-remote bugfix/issue-123
# Split subtree changes into separate repository
git subtree split --prefix=lib/shared-utils -b subtree-changesManaging large monorepos requires specialized tooling to maintain performance, enable selective builds, and provide efficient developer workflows. Enterprise monorepos rely on sophisticated build systems and optimization strategies to scale effectively.
Best for: Large-scale, polyglot monorepos
Features: Incremental builds, remote caching, sandboxed execution
Best for: JavaScript/TypeScript monorepos
Features: Dependency graph, affected detection, distributed caching
Best for: Node.js monorepos with strict dependency management
Features: Phantom dependency detection, incremental publishing
Build observability and remote caching for Bazel
Code search and navigation for large codebases
Enable developers to work with only relevant portions of large repositories
# Enable sparse-checkout
git config core.sparseCheckout true
# Define sparse-checkout patterns
echo "frontend/*" > .git/info/sparse-checkout
echo "shared/components/*" >> .git/info/sparse-checkout
echo "!*/tests/" >> .git/info/sparse-checkout
# Apply sparse checkout
git read-tree -m -u HEADReduce clone time by limiting history depth for CI/CD environments
# Shallow clone with limited history
git clone --depth 1 --single-branch --branch main repo.git
# Deepen history when needed
git fetch --unshallow
# Partial clone (Git 2.19+)
git clone --filter=blob:none repo.gitBuild only changed components and their dependencies
Share build artifacts across team members and CI systems
Single main branch with short-lived feature branches and frequent integration
Run tests only for projects affected by changes, reducing CI time
# Nx affected testing example
nx affected:test --base=main~1 --head=HEAD
# Bazel selective testing
bazel test $(bazel query 'rdeps(//..., //path/to/changed:target)')Transitioning between repository architectures in enterprise environments requires careful planning, phased execution, and comprehensive migration strategies that minimize disruption to ongoing development work.
# Create new monorepo
git init enterprise-monorepo
cd enterprise-monorepo
# Add each repository as subtree
git subtree add --prefix=services/auth https://github.com/company/auth-service.git main
git subtree add --prefix=services/api https://github.com/company/api-service.git main
git subtree add --prefix=frontend/web https://github.com/company/web-frontend.git main
git subtree add --prefix=shared/components https://github.com/company/ui-components.git main# Extract service with full history
git subtree split --prefix=services/auth -b auth-service-history
git clone --branch auth-service-history . ../auth-service-new
# Alternative: filter-repo for complex extractions
git filter-repo --path services/auth/ --to-subdirectory-filter auth-service