Case Studies

We covered the basic workflow in Optimization Guide.

In this section we share miscellaneous performance-related topics we encountered while optimizing production pipelines.

• Parallelism and Performance
  • Pipeline Speed
  • Inconsistent Performance
  • Noisy Neighbour and multiprocessing
• Data Format and Performance
  • The Performance of numpy.load on NPZ data
  • Changing the data format
  • Loading data from memory
  • Practical Example
• Building Inference Pipeline
  • Challenge: Keeping GPU Busy
  • Solution: Software Pipelining with SPDL
  • Running GPU Inference in Background Thread