Update README.md
This commit is contained in:
161
README.md
161
README.md
@@ -1,2 +1,161 @@
|
|||||||
# hello-codeartifact
|
# AWS CodeArtifact Python Package Pipeline
|
||||||
|
|
||||||
|
This repository contains a Jenkins pipeline for building and publishing Python packages to AWS CodeArtifact.
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
This CI/CD pipeline automates the process of building, testing, securing, and publishing Python packages to a private AWS CodeArtifact repository. It ensures consistent builds, security compliance, and reliable package distribution in an enterprise environment.
|
||||||
|
|
||||||
|
## Pipeline Architecture
|
||||||
|
|
||||||
|
### Environment Configuration
|
||||||
|
- Leverages Jenkins Credentials for secure storage of AWS configuration
|
||||||
|
- Uses Docker containers (Python 3.11-slim) for consistent build environments
|
||||||
|
- Configures AWS CodeArtifact as primary package index with PyPI fallback
|
||||||
|
|
||||||
|
### Pipeline Stages
|
||||||
|
|
||||||
|
#### 1. **Checkout**
|
||||||
|
- Retrieves source code from Git repository
|
||||||
|
- Standard SCM checkout process
|
||||||
|
|
||||||
|
#### 2. **Authenticate & Configure**
|
||||||
|
- Generates short-lived AWS CodeArtifact authentication tokens (12-hour expiration)
|
||||||
|
- Constructs secure repository URLs with embedded credentials
|
||||||
|
- Uses AWS IAM roles via Jenkins AWS plugin for authentication
|
||||||
|
|
||||||
|
#### 3. **Build**
|
||||||
|
- Executes within isolated Docker container
|
||||||
|
- Installs build dependencies (setuptools, wheel, twine)
|
||||||
|
- Produces both source distribution (.tar.gz) and wheel (.whl) artifacts
|
||||||
|
- Configures PATH for pip user installations
|
||||||
|
|
||||||
|
#### 4. **Security Scan** (Parallel Execution)
|
||||||
|
- **Trivy Scanner**: Performs filesystem vulnerability analysis
|
||||||
|
- **Safety Check**: Analyzes Python package dependencies
|
||||||
|
- Fails pipeline on HIGH/CRITICAL severity vulnerabilities
|
||||||
|
- Archives all security reports for compliance
|
||||||
|
|
||||||
|
#### 5. **Test**
|
||||||
|
- Installs built package in clean environment
|
||||||
|
- Executes pytest test suite if present
|
||||||
|
- Generates JUnit XML reports for Jenkins integration
|
||||||
|
- Continues pipeline even if tests fail (for visibility)
|
||||||
|
|
||||||
|
#### 6. **Publish**
|
||||||
|
- Uploads package artifacts to AWS CodeArtifact repository
|
||||||
|
- Uses temporary `.pypirc` configuration
|
||||||
|
- Publishes all artifacts from dist/ directory
|
||||||
|
|
||||||
|
### Post-Build Actions
|
||||||
|
- Archives build artifacts for traceability
|
||||||
|
- Cleans workspace to prevent credential leakage
|
||||||
|
- Sends notifications on success/failure (when configured)
|
||||||
|
|
||||||
|
## Technical Implementation
|
||||||
|
|
||||||
|
### Security Features
|
||||||
|
- Temporary authentication tokens minimize credential exposure
|
||||||
|
- Non-root container execution
|
||||||
|
- Parallel security scanning for comprehensive coverage
|
||||||
|
- Workspace cleanup prevents sensitive data persistence
|
||||||
|
|
||||||
|
### Docker Strategy
|
||||||
|
- Each stage runs in isolated container using `docker.image().inside()`
|
||||||
|
- Shared workspace volume for artifact passing between stages
|
||||||
|
- Non-root execution with `HOME=/tmp` configuration
|
||||||
|
|
||||||
|
### Key Design Decisions
|
||||||
|
|
||||||
|
1. **AWS CodeArtifact Integration**
|
||||||
|
- Private package hosting for security
|
||||||
|
- AWS IAM integration for authentication
|
||||||
|
- Automatic PyPI package caching
|
||||||
|
- Enterprise compliance support
|
||||||
|
|
||||||
|
2. **Containerized Builds**
|
||||||
|
- Eliminates Python version conflicts
|
||||||
|
- Ensures reproducible builds
|
||||||
|
- Provides clean build environment
|
||||||
|
|
||||||
|
3. **Parallel Security Scanning**
|
||||||
|
- Reduces overall pipeline execution time
|
||||||
|
- Multiple vulnerability detection methods
|
||||||
|
- Comprehensive security coverage
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
### Required Jenkins Credentials
|
||||||
|
- `AWS_REGION`: AWS region for CodeArtifact
|
||||||
|
- `AWS_ACCOUNT_ID`: AWS account identifier
|
||||||
|
- `CODEART_DOMAIN`: CodeArtifact domain name
|
||||||
|
- `CODEART_REPO`: CodeArtifact repository name
|
||||||
|
- `jenkins-codeartifact`: AWS IAM credentials
|
||||||
|
|
||||||
|
### Pipeline Options
|
||||||
|
```groovy
|
||||||
|
disableConcurrentBuilds() // Prevents parallel execution
|
||||||
|
buildDiscarder(logRotator(numToKeepStr: '50')) // Retains 50 builds
|
||||||
|
timeout(time: 30, unit: 'MINUTES') // 30-minute execution limit
|
||||||
|
```
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### Installing Published Packages
|
||||||
|
```bash
|
||||||
|
# Configure AWS CodeArtifact
|
||||||
|
aws codeartifact login --tool pip \
|
||||||
|
--domain YOUR_DOMAIN \
|
||||||
|
--repository YOUR_REPO \
|
||||||
|
--domain-owner YOUR_ACCOUNT_ID
|
||||||
|
|
||||||
|
# Install package
|
||||||
|
pip install hello-codeartifact
|
||||||
|
```
|
||||||
|
|
||||||
|
### Using the Package
|
||||||
|
```python
|
||||||
|
from hello_pkg import greet
|
||||||
|
|
||||||
|
message = greet()
|
||||||
|
print(message) # Output: Hello, CodeArtifact!
|
||||||
|
```
|
||||||
|
|
||||||
|
## Monitoring and Reporting
|
||||||
|
|
||||||
|
- **Build Artifacts**: Stored in Jenkins for each build
|
||||||
|
- **Test Results**: JUnit XML reports integrated with Jenkins
|
||||||
|
- **Security Reports**: JSON reports from Trivy and Safety
|
||||||
|
- **Build History**: 50 builds retained for audit purposes
|
||||||
|
|
||||||
|
## Future Enhancements
|
||||||
|
|
||||||
|
- Implement semantic versioning automation
|
||||||
|
- Add code coverage reporting
|
||||||
|
- Integrate SBOM (Software Bill of Materials) generation
|
||||||
|
- Implement multi-environment deployment strategies
|
||||||
|
- Add performance benchmarking
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Common Issues
|
||||||
|
- **Authentication Failures**: Verify AWS credentials and IAM permissions
|
||||||
|
- **Build Failures**: Check Docker availability and Python syntax
|
||||||
|
- **Security Scan Failures**: Review vulnerability reports in archived artifacts
|
||||||
|
|
||||||
|
### Debug Commands
|
||||||
|
```bash
|
||||||
|
# Verify CodeArtifact connection
|
||||||
|
aws codeartifact list-packages --domain DOMAIN --repository REPO
|
||||||
|
|
||||||
|
# Test package installation
|
||||||
|
pip install hello-codeartifact --index-url CODEARTIFACT_URL
|
||||||
|
```
|
||||||
|
|
||||||
|
## Contributing
|
||||||
|
|
||||||
|
When modifying the pipeline:
|
||||||
|
1. Test changes in a development branch
|
||||||
|
2. Ensure security scans pass
|
||||||
|
3. Verify package publishes correctly
|
||||||
|
4. Update documentation as needed
|
Reference in New Issue
Block a user