Update README.md
This commit is contained in:
161
README.md
161
README.md
@@ -1,2 +1,161 @@
|
||||
# hello-codeartifact
|
||||
# AWS CodeArtifact Python Package Pipeline
|
||||
|
||||
This repository contains a Jenkins pipeline for building and publishing Python packages to AWS CodeArtifact.
|
||||
|
||||
## Overview
|
||||
|
||||
This CI/CD pipeline automates the process of building, testing, securing, and publishing Python packages to a private AWS CodeArtifact repository. It ensures consistent builds, security compliance, and reliable package distribution in an enterprise environment.
|
||||
|
||||
## Pipeline Architecture
|
||||
|
||||
### Environment Configuration
|
||||
- Leverages Jenkins Credentials for secure storage of AWS configuration
|
||||
- Uses Docker containers (Python 3.11-slim) for consistent build environments
|
||||
- Configures AWS CodeArtifact as primary package index with PyPI fallback
|
||||
|
||||
### Pipeline Stages
|
||||
|
||||
#### 1. **Checkout**
|
||||
- Retrieves source code from Git repository
|
||||
- Standard SCM checkout process
|
||||
|
||||
#### 2. **Authenticate & Configure**
|
||||
- Generates short-lived AWS CodeArtifact authentication tokens (12-hour expiration)
|
||||
- Constructs secure repository URLs with embedded credentials
|
||||
- Uses AWS IAM roles via Jenkins AWS plugin for authentication
|
||||
|
||||
#### 3. **Build**
|
||||
- Executes within isolated Docker container
|
||||
- Installs build dependencies (setuptools, wheel, twine)
|
||||
- Produces both source distribution (.tar.gz) and wheel (.whl) artifacts
|
||||
- Configures PATH for pip user installations
|
||||
|
||||
#### 4. **Security Scan** (Parallel Execution)
|
||||
- **Trivy Scanner**: Performs filesystem vulnerability analysis
|
||||
- **Safety Check**: Analyzes Python package dependencies
|
||||
- Fails pipeline on HIGH/CRITICAL severity vulnerabilities
|
||||
- Archives all security reports for compliance
|
||||
|
||||
#### 5. **Test**
|
||||
- Installs built package in clean environment
|
||||
- Executes pytest test suite if present
|
||||
- Generates JUnit XML reports for Jenkins integration
|
||||
- Continues pipeline even if tests fail (for visibility)
|
||||
|
||||
#### 6. **Publish**
|
||||
- Uploads package artifacts to AWS CodeArtifact repository
|
||||
- Uses temporary `.pypirc` configuration
|
||||
- Publishes all artifacts from dist/ directory
|
||||
|
||||
### Post-Build Actions
|
||||
- Archives build artifacts for traceability
|
||||
- Cleans workspace to prevent credential leakage
|
||||
- Sends notifications on success/failure (when configured)
|
||||
|
||||
## Technical Implementation
|
||||
|
||||
### Security Features
|
||||
- Temporary authentication tokens minimize credential exposure
|
||||
- Non-root container execution
|
||||
- Parallel security scanning for comprehensive coverage
|
||||
- Workspace cleanup prevents sensitive data persistence
|
||||
|
||||
### Docker Strategy
|
||||
- Each stage runs in isolated container using `docker.image().inside()`
|
||||
- Shared workspace volume for artifact passing between stages
|
||||
- Non-root execution with `HOME=/tmp` configuration
|
||||
|
||||
### Key Design Decisions
|
||||
|
||||
1. **AWS CodeArtifact Integration**
|
||||
- Private package hosting for security
|
||||
- AWS IAM integration for authentication
|
||||
- Automatic PyPI package caching
|
||||
- Enterprise compliance support
|
||||
|
||||
2. **Containerized Builds**
|
||||
- Eliminates Python version conflicts
|
||||
- Ensures reproducible builds
|
||||
- Provides clean build environment
|
||||
|
||||
3. **Parallel Security Scanning**
|
||||
- Reduces overall pipeline execution time
|
||||
- Multiple vulnerability detection methods
|
||||
- Comprehensive security coverage
|
||||
|
||||
## Configuration
|
||||
|
||||
### Required Jenkins Credentials
|
||||
- `AWS_REGION`: AWS region for CodeArtifact
|
||||
- `AWS_ACCOUNT_ID`: AWS account identifier
|
||||
- `CODEART_DOMAIN`: CodeArtifact domain name
|
||||
- `CODEART_REPO`: CodeArtifact repository name
|
||||
- `jenkins-codeartifact`: AWS IAM credentials
|
||||
|
||||
### Pipeline Options
|
||||
```groovy
|
||||
disableConcurrentBuilds() // Prevents parallel execution
|
||||
buildDiscarder(logRotator(numToKeepStr: '50')) // Retains 50 builds
|
||||
timeout(time: 30, unit: 'MINUTES') // 30-minute execution limit
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Installing Published Packages
|
||||
```bash
|
||||
# Configure AWS CodeArtifact
|
||||
aws codeartifact login --tool pip \
|
||||
--domain YOUR_DOMAIN \
|
||||
--repository YOUR_REPO \
|
||||
--domain-owner YOUR_ACCOUNT_ID
|
||||
|
||||
# Install package
|
||||
pip install hello-codeartifact
|
||||
```
|
||||
|
||||
### Using the Package
|
||||
```python
|
||||
from hello_pkg import greet
|
||||
|
||||
message = greet()
|
||||
print(message) # Output: Hello, CodeArtifact!
|
||||
```
|
||||
|
||||
## Monitoring and Reporting
|
||||
|
||||
- **Build Artifacts**: Stored in Jenkins for each build
|
||||
- **Test Results**: JUnit XML reports integrated with Jenkins
|
||||
- **Security Reports**: JSON reports from Trivy and Safety
|
||||
- **Build History**: 50 builds retained for audit purposes
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- Implement semantic versioning automation
|
||||
- Add code coverage reporting
|
||||
- Integrate SBOM (Software Bill of Materials) generation
|
||||
- Implement multi-environment deployment strategies
|
||||
- Add performance benchmarking
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
- **Authentication Failures**: Verify AWS credentials and IAM permissions
|
||||
- **Build Failures**: Check Docker availability and Python syntax
|
||||
- **Security Scan Failures**: Review vulnerability reports in archived artifacts
|
||||
|
||||
### Debug Commands
|
||||
```bash
|
||||
# Verify CodeArtifact connection
|
||||
aws codeartifact list-packages --domain DOMAIN --repository REPO
|
||||
|
||||
# Test package installation
|
||||
pip install hello-codeartifact --index-url CODEARTIFACT_URL
|
||||
```
|
||||
|
||||
## Contributing
|
||||
|
||||
When modifying the pipeline:
|
||||
1. Test changes in a development branch
|
||||
2. Ensure security scans pass
|
||||
3. Verify package publishes correctly
|
||||
4. Update documentation as needed
|
Reference in New Issue
Block a user