Appearance
Performance Optimization Tips
Overview
This guide provides comprehensive strategies for optimizing ErgoPlanner performance when working with large models, long simulations, or complex multi-scenario analyses. Learn how to reduce simulation time, manage memory efficiently, and maximize computational throughput.
What You'll Learn:
- Model simplification techniques
- Solver optimization strategies
- Memory management best practices
- Parallel processing capabilities
- Hardware optimization recommendations
- Large-scale simulation workflows
Target Audience:
- Users working with large facilities (>50 MGD)
- Multi-train plant models
- Long-term simulations (>1 year)
- Monte Carlo and optimization studies
- Users experiencing performance issues
Part 1: Performance Fundamentals
1.1 Understanding Computational Complexity
What Affects Simulation Speed:
Primary Factors:
1. Number of state variables
- More tanks = more equations
- Complex models (ASM2d) vs simple (ASM1)
2. Time resolution
- Smaller timestep = more calculations
- 5-min timestep: 105,120 steps/year
- 1-hour timestep: 8,760 steps/year (12× fewer)
3. Simulation duration
- Linear scaling: 2× longer = 2× slower
- Warm-up period often necessary (3-5× SRT)
4. Solver complexity
- Explicit methods: Fast but less accurate
- Implicit methods: Slower but more stable
5. Model structure
- Recycles increase complexity
- Controls add computational overheadTypical Computational Costs:
Relative Simulation Time (baseline = simple AS):
Model Complexity:
- Simple AS (ASM1): 1.0× (baseline)
- BNR (ASM1): 1.5-2.0×
- BNR (ASM2d): 2.5-3.5×
- BNR (ASM3): 2.0-3.0×
- Multi-train with controls: 3.0-5.0×
Timestep Impact (1-year simulation):
- 1 hour: 1.0× (baseline)
- 30 minutes: 2.0×
- 15 minutes: 4.0×
- 5 minutes: 12×
- 1 minute: 60×
Spatial Resolution:
- Single basin per process: 1.0× (baseline)
- 2 basins (parallel): 2.0×
- 4 basins with interconnects: 4.5×
- 8 basins in series/parallel: 9.0×1.2 Performance Benchmarks
Expected Simulation Times (Modern Desktop):
Steady-State Simulations:
- Simple AS (10 MGD): 10-20 seconds
- BNR (10 MGD): 30-60 seconds
- Large multi-train (50 MGD): 2-5 minutes
- Complex multi-facility: 5-15 minutes
Dynamic Simulations (1 year, hourly timestep):
- Simple AS: 3-8 minutes
- BNR (ASM1): 10-20 minutes
- BNR (ASM2d): 20-40 minutes
- Large multi-train: 40-90 minutes
Multi-Scenario Studies (100 scenarios):
- Steady-state screening: 30-60 minutes
- Dynamic validation: 12-24 hours
Monte Carlo (1000 runs):
- Steady-state: 2-6 hours
- Dynamic (simplified): 24-48 hours
If your simulations significantly slower:
→ Optimization needed (this guide!)1.3 Performance Monitoring
Built-in Profiler:
Enable Performance Monitoring:
1. Simulation → Settings → Advanced
2. Check "Enable Performance Profiling"
3. Run simulation
4. View: Results → Performance → Profile Report
Profiler Shows:
- Total simulation time
- Time per iteration
- Time per process unit
- Solver iterations count
- Memory usage
- Bottleneck identificationKey Metrics to Track:
Metric Good Marginal Poor
─────────────────────────────────────────────────
Iterations/step < 5 5-10 > 10
Convergence rate > 95% 85-95% < 85%
Memory usage < 2 GB 2-4 GB > 4 GB
Solver rejections < 1% 1-5% > 5%
Time/step (dynamic) < 0.5s 0.5-2s > 2s
If metrics in "Poor" range → optimization essentialPart 2: Model Simplification
2.1 Reduce Spatial Resolution
When to Simplify:
- Screening studies (design alternatives)
- Sensitivity analysis
- Long-term simulations (>1 year)
- When detailed spatial distribution not critical
Simplification Strategies:
Original Model: 8 Parallel Trains
- Primary clarifiers: 8 units
- Aeration basins: 8 trains (32 tanks total)
- Secondary clarifiers: 8 units
- Total units: 48
Simplified Model: 2 Equivalent Trains
- Primary clarifiers: 2 units (each represents 4 originals)
- Aeration basins: 2 trains (8 tanks total)
- Secondary clarifiers: 2 units
- Total units: 12 (4× reduction)
Speedup: 3-4× faster
Accuracy: >90% for effluent quality predictionSeries Tank Reduction:
Original: 8 Tanks in Series
- Each tank: 0.5 MG
- Total: 4.0 MG
- Detailed spatial profile
Simplified: 2-3 Tanks in Series
- Tanks: 1.33-2.0 MG each
- Total: 4.0 MG (same)
- Reasonable profile approximation
Rule of Thumb:
- For plug flow: N ≥ 3 tanks captures 85% of behavior
- For complete mix: N = 1 adequate
- Intermediate: N = 2-3 usually sufficient
Speedup: 2-4× (for 8→2 reduction)Example:
Original Model:
- Aerobic zone: 8 tanks × 0.75 MG = 6.0 MG total
- Each tank modeled individually
- Simulation time: 45 minutes (1 year dynamic)
Simplified Model:
- Aerobic zone: 3 tanks × 2.0 MG = 6.0 MG total
- Represents same total volume and approximate plug flow
- Simulation time: 18 minutes (2.5× speedup)
Validation:
- Effluent NH4: 0.82 vs 0.79 mg/L (3.8% difference)
- Effluent TN: 6.5 vs 6.3 mg/L (3.2% difference)
- Result: Acceptable for screening study ✓2.2 Use Simpler Biological Models
Model Complexity Hierarchy:
Model State Vars Relative Speed Use Case
──────────────────────────────────────────────────────────
Simple AS 4-6 1.0× (fastest) Organic removal only
ASM1 13 1.5× C, N removal
ASM2 19 3.0× C, N, P removal
ASM2d 19 3.2× C, N, P (denitrifying PAO)
ASM3 13 2.5× Storage, C, N removal
Recommendation:
- Use simplest model that captures required processes
- Don't use ASM2d if P removal not modeled
- ASM1 adequate for many BNR applicationsWhen to Downgrade Model:
ASM2d → ASM1:
☑ If bio-P not present or using chemical P removal
☑ Screening studies where P details not critical
☑ Long-term simulations (>2 years)
Result: 2-3× speedup
ASM3 → ASM1:
☑ If storage dynamics not critical
☑ Steady-state or slowly varying conditions
Result: 1.5-2× speedup
Custom → Standard Model:
☑ Use built-in models instead of user-defined
☑ Built-in models are optimized for performance
Result: 1.2-1.5× speedup2.3 Remove Unnecessary Components
Streamline Model:
Components Often Removable:
☐ Minor pretreatment units (screens, grit chambers)
→ Impact: Minimal on biological treatment
→ Speedup: 5-10%
☐ Detailed primary clarifier model
→ Use simple removal efficiency instead
→ Speedup: 10-15%
☐ Tertiary processes (if not critical)
→ Filtration, disinfection
→ Speedup: 5-10% per unit
☐ Detailed sludge processing
→ Use simple mass balance
→ Speedup: 15-25%
☐ Administrative buildings, vehicles
→ Model only core treatment
→ Speedup: Variable
Total Potential Speedup: 40-60%Example Streamlining:
Original Full-Plant Model:
- Headworks (screens, grit)
- Primary clarifiers (detailed)
- BNR (detailed)
- Secondary clarifiers (detailed)
- Filtration
- UV disinfection
- Thickening
- Anaerobic digestion
- Dewatering
- Total simulation time: 85 minutes
Streamlined Core Model:
- Influent (characterized)
- BNR (detailed)
- Secondary clarifiers (detailed)
- Effluent (monitored)
- WAS (simple mass balance)
- Total simulation time: 32 minutes
Speedup: 2.7×
Use case: Focus on BNR optimizationPart 3: Solver Optimization
3.1 Select Appropriate Solver
Solver Types:
Explicit Methods (Euler, RK4):
Pros:
- Fast per iteration
- Simple implementation
- Good for non-stiff systems
Cons:
- Require small timestep for stability
- Not suitable for stiff systems (fast + slow dynamics)
Use for:
- Simple models
- Short simulations
- Non-stiff problems
Implicit Methods (BDF, Rosenbrock):
Pros:
- Handle stiff systems well
- Allow larger timesteps
- More stable
Cons:
- Slower per iteration
- More complex
Use for:
- Complex biological models
- Long simulations
- Systems with fast reactions (nitrification)
Adaptive Methods (CVODE, LSODA):
Pros:
- Automatically adjust timestep
- Balance speed and accuracy
- Handle variable stiffness
Cons:
- Overhead for adaptation
- May be slower for simple problems
Use for:
- General purpose (recommended)
- Unknown stiffness
- Variable dynamicsSolver Selection Guide:
Your Situation Recommended Solver
────────────────────────────────────────────────────────────────
Simple AS, steady-state Euler or RK4
BNR, steady-state BDF or Rosenbrock
Any model, dynamic (<1 month) RK4 or CVODE
Any model, dynamic (>1 month) BDF or LSODA (adaptive)
Very stiff (nitrification, controls) BDF or Rosenbrock
Uncertain CVODE (adaptive)Switching Solvers:
In ErgoPlanner:
1. Simulation → Settings → Advanced → Solver
2. Select from dropdown:
- Euler (explicit)
- RK4 (Runge-Kutta 4th order)
- BDF (Backward Differentiation Formula)
- Rosenbrock
- CVODE (adaptive)
- LSODA (adaptive)
3. Test with short simulation
4. Use fastest solver that converges reliablyExample:
Model: BNR (ASM1), 1-year simulation, hourly timestep
Test Results:
Solver Time Convergence Notes
───────────────────────────────────────────────────────
Euler 25 min Failed Unstable
RK4 38 min OK Baseline
BDF 22 min OK Faster!
Rosenbrock 28 min OK Good
CVODE 26 min OK Adaptive
LSODA 24 min OK Best balance
Selected: BDF (fastest reliable option)
Speedup vs baseline (RK4): 1.7×3.2 Optimize Tolerances
Tolerance Settings:
Absolute Tolerance (atol):
- Minimum acceptable error (concentration units)
- Typical: 1e-6 to 1e-8 mg/L
- Tighter (smaller) = slower but more accurate
Relative Tolerance (rtol):
- Relative error as fraction
- Typical: 1e-4 to 1e-6
- Tighter (smaller) = slower but more accurate
Trade-off:
- Tight tolerances: Accurate but slow
- Loose tolerances: Fast but may miss dynamicsRecommended Settings:
Application atol rtol Priority
──────────────────────────────────────────────────────────
Screening studies 1e-5 1e-3 Speed
Design (preliminary) 1e-6 1e-4 Balanced
Design (final) 1e-7 1e-5 Accuracy
Calibration 1e-7 1e-5 Accuracy
Optimization (many runs) 1e-6 1e-4 Balanced
Monte Carlo (1000+ runs) 1e-5 1e-3 Speed
Regulatory compliance 1e-8 1e-6 AccuracyRelaxing Tolerances:
Example: Monte Carlo with 1000 runs
Original Settings:
- atol = 1e-7
- rtol = 1e-5
- Time per run: 45 minutes
- Total time: 750 hours (31 days!)
Relaxed Settings:
- atol = 1e-5 (100× looser)
- rtol = 1e-3 (100× looser)
- Time per run: 18 minutes (2.5× faster)
- Total time: 300 hours (12.5 days)
Accuracy Impact:
- Effluent TN: 6.52 vs 6.48 mg/L (0.6% difference)
- Effluent TP: 0.84 vs 0.83 mg/L (1.2% difference)
- Result: Acceptable for statistical analysis ✓
Recommendation: Relax tolerances for large studies3.3 Adjust Maximum Iterations
Iteration Limits:
Default: 100 iterations per timestep
Too Low:
- Model may not converge
- Results inaccurate
- Warning messages
Too High:
- Wasted computation if convergence not possible
- Slow simulation
- May indicate problem elsewhere
Optimal:
- Most steps converge in 3-10 iterations
- Occasional steps use 10-50 iterations
- Rarely hit maximumTuning Iterations:
Check Iteration Statistics:
Results → Convergence → Iteration Histogram
Good Distribution:
Iterations Frequency
1-5 75%
6-10 18%
11-20 5%
21-50 2%
>50 <1%
If >5% of steps hit maximum:
→ Increase max iterations (100 → 200)
→ Or address root cause (tolerances, timestep, model)
If <1% use >20 iterations:
→ Can reduce max iterations (100 → 50)
→ Saves computation on failed convergencePart 4: Time Step Optimization
4.1 Select Appropriate Time Step
Time Step Selection Guide:
Process Time Scale Suggested Timestep
────────────────────────────────────────────────
Steady-state N/A (equilibrium solver)
Slow dynamics (SRT>15d) 1-4 hours
Typical AS (SRT=10d) 15-60 minutes
Fast nitrification 5-15 minutes
Process control 1-5 minutes
Detailed diurnal 15 minutes
Storm events 5-15 minutes
Annual simulation 1 hour (typical)
Multi-year 2-4 hoursAdaptive Time Stepping:
Instead of fixed timestep, use adaptive:
Advantages:
- Small steps during fast changes (storms, upsets)
- Large steps during steady periods
- Automatic balancing of speed and accuracy
Enable in ErgoPlanner:
1. Simulation → Settings → Time Step Mode
2. Select "Adaptive"
3. Set:
- Minimum timestep: 1-5 minutes
- Maximum timestep: 1-4 hours
- Initial timestep: 15-30 minutes
Typical Speedup: 1.5-3× for annual simulationsExample:
1-Year Simulation, Fixed Timestep:
15-minute timestep:
- Total steps: 35,040
- Simulation time: 68 minutes
- Very detailed results
1-hour timestep:
- Total steps: 8,760
- Simulation time: 18 minutes (3.8× faster)
- Adequate detail for most purposes
4-hour timestep:
- Total steps: 2,190
- Simulation time: 6 minutes (11× faster)
- Missing some diurnal dynamics
- OK for screening only
Adaptive (15 min to 2 hr):
- Avg timestep: ~45 minutes
- Total steps: ~12,000
- Simulation time: 22 minutes (3× faster than fixed 15-min)
- Captures dynamics when needed
- Uses large steps during stable periods
- Best of both worlds ✓4.2 Reduce Simulation Duration
Strategies:
1. Steady-State Instead of Dynamic
- If only equilibrium results needed
- Speedup: 10-100× faster
- Use: Design, capacity analysis
2. Shorten Warm-Up Period
- Default: 3-5× SRT
- If good initial conditions: 1-2× SRT
- Speedup: 2-3× for long-SRT systems
3. Analyze Critical Periods Only
- Summer vs winter (not full year)
- Peak week vs average week
- Design storm vs continuous
- Speedup: 10-50× (depending on period)
4. Use Representative Days
- Typical day (average)
- Peak day
- Minimum day
- 3 simulations instead of 365
- Speedup: 100× (but less comprehensive)Example:
Original Plan: 2-year dynamic simulation
- Purpose: Assess seasonal performance
- Duration: 730 days
- Timestep: 1 hour
- Simulation time: 95 minutes
Optimized Approach:
1. Steady-state at key conditions:
- Summer average: 2 minutes
- Summer peak: 2 minutes
- Winter average: 2 minutes
- Winter minimum: 2 minutes
Total: 8 minutes
2. Dynamic validation:
- 1 summer month: 8 minutes
- 1 winter month: 8 minutes
Total: 16 minutes
Total Time: 24 minutes (4× faster than original)
Coverage:
- Key operating conditions: ✓
- Seasonal variation: ✓
- Dynamic validation: ✓
- Sufficient for design decisionsPart 5: Memory Management
5.1 Understand Memory Usage
Memory Consumers:
1. Model State
- State variables per tank/process
- Increases with spatial resolution
- Typical: 1-100 MB
2. Input Data
- Influent time series
- 1 year hourly = 8,760 points × parameters
- Typical: 1-10 MB
3. Results Storage
- Output variables × timesteps
- Can become very large
- Typical: 10-1000 MB
4. Solver Workspace
- Temporary arrays for calculations
- Scales with model size
- Typical: 10-100 MB
5. Graphics/UI
- Real-time plotting
- Preview rendering
- Typical: 50-200 MB
Total: Usually 100-2000 MB
Large models: Up to 4-8 GBCheck Memory Usage:
Task Manager (Windows) or Activity Monitor (Mac):
- Monitor "ErgoPlanner" process
- Watch during simulation
- Note peak usage
In ErgoPlanner:
- Help → System Information → Memory Usage
- Results → Performance → Memory Profile5.2 Reduce Memory Footprint
Strategies:
1. Reduce Output Frequency
Original: Save every timestep
- 1-year, 15-min step = 35,040 points
- Memory: ~500 MB
Optimized: Save every hour
- 1-year = 8,760 points
- Memory: ~125 MB (4× reduction)
Settings:
Simulation → Output → Save Interval → 1 hour
2. Select Output Variables
Original: Save all variables (50+)
- Memory: 500 MB
Optimized: Save only needed (10)
- Effluent quality
- MLSS
- DO
- Key indicators
- Memory: ~100 MB (5× reduction)
Settings:
Simulation → Output → Select Variables
3. Clear Results Between Runs
- For multi-scenario studies
- Clear old results before new run
- Settings → Clear Results After Export
4. Use Results Database
- Store results in database, not RAM
- Slower access but unlimited size
- Settings → Advanced → Use Database Storage
5. Disable Real-Time Plotting
- Plotting consumes memory
- Disable during batch runs
- Settings → Display → Disable Live PlotsExample:
Monte Carlo: 1000 Runs
Original Memory Usage:
- Save all variables, every timestep
- Keep all results in memory
- Memory per run: 800 MB
- Total: 800 GB (impossible!)
- Crashes after ~5 runs
Optimized:
1. Save 10 key variables only
2. Save hourly (not every 15 min)
3. Export results to CSV after each run
4. Clear memory before next run
5. Disable live plotting
Memory per run: 150 MB
Total in RAM: 150 MB (reused)
Results: Exported to disk (50 GB total, manageable)
Result: All 1000 runs complete successfully ✓5.3 Handle Large Datasets
Large Input Data:
Problem: 10 years of 5-minute data
- 1,051,200 data points
- File size: 500 MB
- Memory: 2 GB when loaded
- Import time: 10 minutes
Solutions:
1. Resample Data
- Aggregate to hourly averages
- 87,600 points (12× reduction)
- Memory: 170 MB
- Import: 30 seconds
- Data → Preprocessing → Resample
2. Split Into Segments
- Load 1 year at a time
- Simulate sequentially
- Export results incrementally
3. Use Database
- Store data in SQLite or similar
- ErgoPlanner queries as needed
- Minimal memory footprint
- Data → Import → From Database
4. Statistical Representation
- Instead of all data, use:
• Mean + std dev
• Diurnal pattern
• Seasonal pattern
- Generate synthetic data
- Tiny memory footprintPart 6: Parallel Processing
6.1 Multi-Scenario Parallel Execution
Parallel Capabilities:
ErgoPlanner can run multiple scenarios simultaneously:
- Each scenario in separate thread
- Utilizes multi-core CPUs
- Near-linear speedup (up to # cores)
Example:
- 100 scenarios
- Each takes 10 minutes
- Sequential: 1000 minutes (16.7 hours)
With 8-core CPU:
- 8 scenarios in parallel
- Total time: 125 minutes (2.1 hours)
- Speedup: 8× (perfect scaling)Enable Parallel Processing:
1. Scenarios → Select Multiple (Ctrl+Click)
2. Right-click → Run in Parallel
3. Settings:
- Max threads: Auto (uses all cores) or specify
- Priority: Normal (recommended)
4. Monitor: Progress bar shows all scenarios
Tips:
- Leave 1-2 cores for OS (don't use 100%)
- Close other applications
- Monitor CPU temperature (cooling)Optimal Thread Count:
CPU Cores Scenarios Recommended Threads
──────────────────────────────────────────────
4 <10 3
4 >10 4
8 <20 6
8 >20 8
16 Any 12-14
32 Large study 24-28
Rule: Threads = Cores - 2 (leave headroom)6.2 Distributed Computing
For Very Large Studies:
ErgoPlanner Server Edition:
- Distribute scenarios across multiple computers
- Network-based job queue
- Aggregate results centrally
Workflow:
1. Define scenarios on main computer
2. Export scenario package
3. Distribute to worker computers
4. Workers run assigned scenarios
5. Return results to main computer
6. Aggregate and analyze
Speedup: N× (where N = number of computers)
Use Cases:
- Monte Carlo with 10,000+ runs
- Comprehensive optimization
- Parameter space exploration
- Sensitivity analysis gridsPart 7: Hardware Optimization
7.1 CPU Selection
CPU Characteristics for ErgoPlanner:
Priority Ranking:
1. Core count (for parallel scenarios)
2. Single-thread performance (for individual simulations)
3. Cache size (for large models)
4. Memory bandwidth
Recommendations:
Budget (<$500):
- AMD Ryzen 5 / Intel i5
- 6-8 cores
- Good for typical users
Mid-Range ($500-$1500):
- AMD Ryzen 7/9 / Intel i7/i9
- 8-16 cores
- Excellent for power users
High-End (>$1500):
- AMD Threadripper / Intel Xeon
- 16-64 cores
- For large-scale studies, professional use
Speedup Expectations:
Budget → Mid-Range: 1.5-2×
Mid-Range → High-End: 2-3× (parallel), 1.2× (single)7.2 Memory (RAM)
RAM Requirements:
Minimum: 8 GB
- Basic models
- Small facilities
- Limited scenarios
Recommended: 16-32 GB
- Typical professional use
- Multi-scenario studies
- Large facilities
High-Performance: 64-128 GB
- Very large models (>100 MGD)
- Monte Carlo (thousands of runs)
- Keep multiple results in memory
Extreme: 256+ GB
- Distributed computing server
- Database applications
- Rarely needed for modeling alone
Guideline:
RAM (GB) ≥ 2 × Largest_Model_Size (GB) + 8 GB (OS)7.3 Storage
Storage Recommendations:
Type Speed Capacity Use Case
──────────────────────────────────────────────────────
SSD (SATA) Good 500GB-2TB Standard
SSD (NVMe) Excellent 1-4TB Recommended
HDD (7200rpm) Fair 4-8TB Data archive
HDD (5400rpm) Poor - Avoid
ErgoPlanner Benefits from Fast Storage:
- Model loading: 2-10× faster (SSD vs HDD)
- Results writing: 3-5× faster
- Data import: 2-4× faster
- Overall: 10-20% simulation time reduction
Recommendation:
- Install ErgoPlanner on SSD
- Store active projects on SSD
- Archive old projects to HDD7.4 System Configuration
Optimization Checklist:
☑ Close unnecessary applications during simulation
☑ Disable antivirus scanning of ErgoPlanner directory
☑ Set ErgoPlanner process priority to "High"
☑ Ensure adequate cooling (prevent thermal throttling)
☑ Use power plan: "High Performance" (not balanced)
☑ Disable sleep/hibernate during long simulations
☑ Keep system drivers updated
☑ Use 64-bit OS and 64-bit ErgoPlanner
☑ Allocate adequate virtual memory (page file)
Expected Improvement: 10-30% faster overallPart 8: Workflow Optimization
8.1 Efficient Calibration Workflow
Staged Calibration:
Stage 1: Coarse Screening (Fast)
- Simplified model (fewer tanks)
- Loose tolerances
- Wide parameter ranges
- Many scenarios (100+)
- Identify promising parameter regions
- Time: 1-2 hours
Stage 2: Refined Search (Moderate)
- Medium complexity model
- Moderate tolerances
- Narrow parameter ranges (from Stage 1)
- Moderate scenarios (20-30)
- Refine parameter estimates
- Time: 2-4 hours
Stage 3: Final Calibration (Detailed)
- Full complexity model
- Tight tolerances
- Fine-tuning (5-10 scenarios)
- Validate with all data
- Time: 1-2 hours
Total: 4-8 hours
vs. Brute Force (full complexity, many scenarios): 20-40 hours
Speedup: 3-5×8.2 Batch Processing
Automated Workflows:
ErgoPlanner Scripting (Python API):
Example: Parameter Sweep
import ergoplanner as ep
# Load base model
model = ep.load_model("BaseModel.eplr")
# Define parameter ranges
srt_values = range(8, 20, 2) # 8, 10, 12, 14, 16, 18
do_values = [1.5, 2.0, 2.5, 3.0]
# Run all combinations
results = []
for srt in srt_values:
for do in do_values:
model.set_parameter("SRT", srt)
model.set_parameter("DO_setpoint", do)
result = model.run_simulation()
results.append({
'SRT': srt,
'DO': do,
'Effluent_TN': result.effluent_TN,
'Energy': result.energy_consumption
})
# Export results
ep.export_csv(results, "ParameterSweep.csv")
Benefits:
- Runs overnight (unattended)
- Consistent methodology
- Easy to repeat
- Comprehensive documentation8.3 Results Management
Efficient Storage:
1. Export Key Results Only
- Don't keep full timeseries for all scenarios
- Export summary statistics:
• Mean, median, 95th percentile
• Min, max
• Compliance rate
- Full data only for selected scenarios
2. Compress Old Projects
- ZIP completed projects
- 5-10× size reduction
- Archive to secondary storage
3. Use Databases for Large Studies
- SQLite for local
- PostgreSQL for shared/server
- Query-based analysis
- Minimal memory footprint
4. Incremental Exports
- Export results batch by batch
- Don't accumulate in memory
- CSV append modePart 9: Optimization Case Studies
9.1 Case Study 1: Large Multi-Train Facility
Challenge:
Facility: 80 MGD, 8 parallel trains
Purpose: 20-year master plan evaluation
Scenarios: 50 design alternatives
Original approach: Full detailed model for each
Estimated Time:
- Model: 8 trains × 4 basins × 3 zones = 96 tanks
- Simulation time: 3 hours per scenario
- Total: 150 hours (6.25 days)Optimization Applied:
1. Model Simplification
- Represented 8 trains as 2 equivalent trains
- Reduced to 24 tanks (4× reduction)
- Speedup: 3×
2. Staged Analysis
- Stage 1: Steady-state screening (all 50)
• 15 minutes per scenario
• 12.5 hours total
• Down-selected to 10 alternatives
- Stage 2: Dynamic validation (10 best)
• 1 hour per scenario
• 10 hours total
3. Parallel Processing
- 8-core CPU
- Run 6 scenarios simultaneously
- Speedup: 5× (accounting for overhead)
Final Time:
- Screening: 12.5 hours / 5 = 2.5 hours
- Validation: 10 hours / 5 = 2 hours
- Total: 4.5 hours
Overall Speedup: 150 / 4.5 = 33×
Result: Completed in half a day instead of week ✓9.2 Case Study 2: Monte Carlo Uncertainty Analysis
Challenge:
Purpose: Assess design reliability under uncertainty
Runs: 1000 Monte Carlo simulations
Model: BNR (ASM2d), 1-year dynamic
Original Setup:
- Full complexity
- 15-minute timestep
- All variables saved
- Time per run: 85 minutes
- Total: 85,000 minutes (59 days!)Optimization Applied:
1. Model: Switched ASM2d → ASM1
- P removal not critical for reliability assessment
- Speedup: 2.5×
2. Timestep: 15 min → 1 hour
- Adequate for annual statistics
- Speedup: 3×
3. Tolerances: Relaxed
- atol: 1e-7 → 1e-5
- rtol: 1e-5 → 1e-3
- Speedup: 1.8×
4. Output: Reduced
- Save only effluent quality
- Hourly output (not every step)
- Export and clear after each run
- Speedup: 1.3× (memory management)
5. Parallel: 12-core workstation
- Run 10 simulations simultaneously
- Speedup: 9× (accounting for overhead)
Cumulative Speedup: 2.5 × 3 × 1.8 × 1.3 × 9 = 88×
Final Time:
- 85 minutes → 58 seconds per run
- 1000 runs: 967 minutes (16 hours)
Result: Completed overnight instead of 2 months ✓
Accuracy Impact:
- Effluent TN 95th percentile: 8.92 vs 8.88 mg/L (0.4% diff)
- Acceptable for reliability analysis9.3 Case Study 3: Real-Time Optimization
Challenge:
Purpose: Daily operational optimization
Requirement: Results within 30 minutes
Model: 25 MGD BNR facility
Task: Find optimal DO and RAS setpoints for next 24 hours
Original Approach:
- Full dynamic simulation
- Test 100 combinations
- Time: 6 hours (too slow for daily use)Optimization Applied:
1. Simplified Model
- Reduced spatial resolution (12 → 4 tanks)
- Speedup: 2×
2. Surrogate Model
- Train neural network on 1000 simulations
- Use NN for fast prediction during optimization
- Full simulation only for final validation
- Speedup: 100× (for optimization iterations)
3. Smart Initial Guess
- Use previous day's optimal solution
- Reduces optimization iterations
- Speedup: 2×
4. Parallel Evaluation
- Genetic algorithm with parallel fitness evaluation
- 4-core CPU
- Speedup: 3×
Total Workflow:
1. Load yesterday's data: 1 minute
2. Run optimization (surrogate model): 10 minutes
3. Validate with full simulation: 5 minutes
4. Generate report: 2 minutes
5. Total: 18 minutes ✓
Result: Operators receive optimal setpoints each morning
Benefit: 5-8% energy savings, improved reliabilityPart 10: Summary and Checklist
10.1 Quick Optimization Checklist
Before Running Large Studies:
☐ Model Simplification
☐ Reduce spatial resolution where possible
☐ Use simplest adequate biological model
☐ Remove unnecessary components
☐ Solver Settings
☐ Select appropriate solver for problem type
☐ Relax tolerances for screening studies
☐ Use adaptive timestepping
☐ Time Management
☐ Use largest acceptable timestep
☐ Run steady-state when appropriate
☐ Minimize warm-up period
☐ Analyze critical periods instead of full duration
☐ Memory Management
☐ Reduce output frequency
☐ Select only needed output variables
☐ Clear results between runs
☐ Disable live plotting for batch jobs
☐ Parallel Processing
☐ Enable multi-threading for scenario batches
☐ Set appropriate thread count
☐ Monitor CPU usage
☐ Hardware
☐ Close unnecessary applications
☐ Use SSD storage
☐ Adequate RAM available
☐ Set high-performance power plan
☐ Workflow
☐ Use staged analysis (coarse → fine)
☐ Script repetitive tasks
☐ Export results incrementally10.2 Expected Speedups
Cumulative Impact of Optimizations:
Optimization Speedup Cumulative
───────────────────────────────────────────────────────────
Baseline (no optimization) 1.0× 1.0×
Reduce spatial resolution (8→2) 3.0× 3.0×
Use simpler model (ASM2d→ASM1) 2.5× 7.5×
Increase timestep (15min→1hr) 3.0× 22.5×
Relax tolerances 1.8× 40.5×
Optimize solver 1.5× 60.8×
Reduce output 1.3× 79.0×
Parallel (8 cores) 6.0× 474×
Realistic Total: 100-500× speedup possible!
Example:
- Original: 100 scenarios × 2 hours = 200 hours (8.3 days)
- Optimized: 200 hours / 300 = 0.67 hours (40 minutes)10.3 When to Optimize
Optimization Decision Matrix:
Situation Action
─────────────────────────────────────────────────────────
Single simulation, <10 minutes No optimization needed
Single simulation, >1 hour Apply basic optimizations
<10 scenarios Basic optimizations
10-100 scenarios Moderate optimization + parallel
>100 scenarios Aggressive optimization
Monte Carlo (>500 runs) Maximum optimization essential
Daily operations Build optimized workflow
Exploratory analysis Fast simplified model
Final design Detailed model, accept longer time10.4 Optimization Best Practices
Do's:
✓ Profile performance before optimizing (identify bottleneck)
✓ Optimize in stages (biggest impact first)
✓ Validate simplified models against detailed
✓ Document optimization trade-offs
✓ Test convergence after tolerance changes
✓ Monitor resource usage (CPU, RAM)
✓ Save different model versions (detailed vs simplified)
✓ Use parallel processing for independent scenarios
✓ Export results incrementally for large studies
✓ Keep detailed model for final validationDon'ts:
✗ Don't over-simplify (know your accuracy requirements)
✗ Don't use loose tolerances without validation
✗ Don't ignore convergence warnings
✗ Don't run detailed model for screening
✗ Don't keep all results in memory
✗ Don't forget to benchmark (time before/after)
✗ Don't optimize prematurely (small studies often fine)
✗ Don't sacrifice accuracy for speed without justificationConclusion
Performance optimization is essential for large-scale modeling studies. By applying the techniques in this guide, you can achieve speedups of 10-500× while maintaining acceptable accuracy.
Key Takeaways:
- Simplify Thoughtfully: Use the simplest model that meets your needs
- Choose Wisely: Right solver and timestep for your problem
- Manage Memory: Don't let memory limit your studies
- Parallelize: Utilize modern multi-core CPUs
- Stage Analysis: Coarse screening → detailed validation
- Invest in Hardware: Good CPU and SSD pay dividends
- Automate Workflows: Scripting saves time on repetitive tasks
- Balance Trade-offs: Speed vs. accuracy depends on application
Next Steps:
- Profile your current workflow
- Apply relevant optimizations from this guide
- Benchmark improvements
- Share successful strategies with team
- Consider hardware upgrades if doing large-scale work regularly
Related Guides:
- Model Calibration
- Troubleshooting Guide
- Best Practices Overview
Questions or optimization tips to share?
Contact ErgoPlanner support or contribute to the community forum.
Last Updated: January 2025 Version: 1.0