Announcer The following program features simulated voices generated for educational and technical exploration.

Sam Dietrich Good evening. I'm Sam Dietrich.

Kara Rousseau And I'm Kara Rousseau. Welcome to Simulectics Radio.

Kara Rousseau General-purpose programming languages aim for universality—write once, run anywhere, express any algorithm. This generality comes at a cost. Compilers must be conservative about optimizations because they can't assume much about program structure or intent. Programmers must explicitly manage low-level details like memory layout, parallelism, and hardware-specific features. Domain-specific languages take a different approach. By restricting expressiveness to a particular computational domain, they enable aggressive optimization that would be impossible or impractical in general-purpose languages. The compiler can exploit domain knowledge to generate highly efficient code automatically. But this creates a fundamental trade-off. You gain performance and productivity within the domain while losing the ability to express computations outside it. Tonight we're examining this trade-off through the lens of image processing and computational photography, where DSLs like Halide have demonstrated dramatic performance improvements.

Sam Dietrich From a hardware perspective, DSLs can be particularly valuable because they expose structure that maps well to specialized architectures. Image processing pipelines, for instance, have regular data access patterns, abundant parallelism, and predictable memory behavior—exactly the properties that accelerators and SIMD units exploit. A general-purpose language forces the programmer to manually orchestrate this mapping, introducing both complexity and opportunities for error. A well-designed DSL can automate this, generating code that saturates memory bandwidth, exploits data locality, and parallelizes across available compute units without requiring the programmer to think about these details explicitly. The question is whether the constraints imposed by domain specialization are acceptable, or whether the need to escape the DSL for even slightly unusual computations undermines its value.

Kara Rousseau Joining us to explore these questions is Dr. Jonathan Ragan-Kelley, an associate professor of Computer Science at MIT. Dr. Ragan-Kelley is one of the creators of Halide, a domain-specific language for image processing that separates algorithmic specification from optimization strategy. His work examines how domain constraints enable compiler optimizations that are impractical for general-purpose languages, and how to design abstractions that preserve programmer productivity while achieving near-hardware performance. Dr. Ragan-Kelley, welcome.

Dr. Jonathan Ragan-Kelley Thank you. I'm glad to discuss these ideas.

Kara Rousseau Let's start with the fundamental design principle of Halide—separating algorithm from schedule. What problem does this separation solve?

Dr. Jonathan Ragan-Kelley In traditional programming, you write code that specifies both what to compute and how to compute it—the algorithm and its execution strategy are intertwined. For image processing, this creates a tension. The natural way to express an algorithm is often terrible for performance. You might write a sequence of operations where each produces a full intermediate image, but materializing those intermediates consumes enormous memory bandwidth and cache capacity. The efficient implementation fuses operations, tiles data to fit in cache, and vectors across SIMD lanes. But expressing this directly in code obscures the algorithm and makes it difficult to change optimization strategies for different architectures or inputs. Halide separates these concerns. You write the algorithm in a pure functional style that describes what values to compute at each point. Separately, you write a schedule that describes how to compute them—what order to evaluate operations, how to tile and parallelize, which intermediates to store versus recompute. The compiler uses the schedule to generate optimized code while verifying that it preserves the algorithm's semantics.

Sam Dietrich How does the compiler verify that a schedule preserves semantics? Reordering and fusion can be subtle, especially with dependencies between pipeline stages.

Dr. Jonathan Ragan-Kelley Halide uses a formal model of dependencies based on the algorithm's structure. Each function defines what values it depends on through its mathematical specification. The compiler analyzes these dependencies to construct a dependence graph. When you apply a schedule transformation—say, fusing two stages or changing iteration order—the compiler checks whether the transformation respects data dependencies. For example, if you want to compute multiple operations on the same tile of data before moving to the next tile, the compiler verifies that later operations don't depend on values from other tiles that haven't been computed yet. This is possible because Halide algorithms are pure functions with explicit domain specifications. The compiler knows exactly what each function reads and writes. In a general-purpose imperative language with arbitrary pointer aliasing and side effects, such analysis would be intractable. The domain restriction enables tractable verification of correctness-preserving transformations.

Kara Rousseau This sounds similar to how loop optimizers in general-purpose compilers work—polyhedral analysis and transformation. What does Halide's domain specificity buy you beyond what a sufficiently smart compiler could do?

Dr. Jonathan Ragan-Kelley Polyhedral compilation is powerful but limited in practice. It requires perfectly nested loops with affine index expressions, which excludes many real programs. More fundamentally, it's automatic—the compiler chooses transformations based on heuristics. This works reasonably well for some workloads but fails unpredictably for others. The programmer has limited control over what optimizations are applied. Halide inverts this. The algorithm is guaranteed to be analyzable because it's expressed in a restricted form. The schedule gives the programmer explicit control over optimization strategy. This is critical because optimal strategy depends on context that the compiler can't see—the target architecture, input characteristics, whether latency or throughput matters. In production image processing pipelines, we routinely need different schedules for CPU versus GPU, for small versus large images, for real-time processing versus offline rendering. Halide makes these trade-offs explicit rather than opaque compiler heuristics. The separation of concerns also makes it possible to search the space of schedules systematically, which we've explored with autoschedulers that use machine learning to find good strategies.

Sam Dietrich What performance improvements have you seen in practice compared to hand-optimized C or assembly code?

Dr. Jonathan Ragan-Kelley It depends on the baseline. Compared to naive C implementations—say, materializing full intermediate images—Halide can be orders of magnitude faster because it fuses operations and exploits locality. Compared to carefully hand-optimized code, Halide is typically competitive or somewhat faster, while requiring far less development effort. For example, in Adobe's image processing pipelines, Halide-generated code matches or exceeds their hand-tuned implementations in performance while being much easier to maintain and retarget across architectures. The real win isn't raw performance but productivity. Writing a highly optimized image processing pipeline by hand requires deep expertise in SIMD intrinsics, cache optimization, threading, and architecture-specific tuning. A typical implementation might take weeks or months. In Halide, you express the algorithm cleanly and then experiment with different schedules, often finding good strategies in days. This makes it practical to explore design space and adapt to new architectures without rewriting everything.

Kara Rousseau What happens when you need to express something outside Halide's computational model? How do you handle escape hatches without undermining the DSL's guarantees?

Dr. Jonathan Ragan-Kelley This is the fundamental tension in DSL design. Halide is designed for pipelines of stencil computations—operations where output at each point depends on a neighborhood of input points. This covers a lot of image processing, but not everything. You can't express arbitrary control flow, dynamic recursion, or irregular data structures directly in Halide. When you need these, you have a few options. One is to implement that piece in a general-purpose language and interface with Halide for the rest. Halide can call external functions and can be called from C++, Python, or other languages. This preserves Halide's optimization properties for the parts that fit its model while allowing escape for parts that don't. Another approach is to extend the DSL to support additional patterns while maintaining analyzability. We've added support for things like reductions, histograms, and scattering operations that don't fit the pure stencil model but are common enough to warrant special support. The key is to maintain the property that the compiler can reason about data dependencies, even if the computational model becomes more complex.

Sam Dietrich How well does Halide map to different hardware architectures—CPUs, GPUs, specialized accelerators?

Dr. Jonathan Ragan-Kelley Retargeting is a core design goal. The same Halide algorithm can generate code for CPUs, CUDA GPUs, OpenCL devices, and even specialized accelerators. The schedule specifies target-specific optimization strategy. For CPUs, you might tile for cache, vectorize across SIMD lanes, and parallelize across cores. For GPUs, you map operations to thread blocks and threads, manage shared memory explicitly, and exploit massive parallelism. The algorithm remains unchanged; only the schedule changes. In practice, finding a good schedule for each target requires expertise. CPU schedules prioritize cache locality and avoid memory bandwidth bottlenecks. GPU schedules maximize occupancy and hide memory latency through parallelism. This isn't automatic—you need to understand the target architecture—but the burden is primarily in the schedule, not in rewriting the algorithm. We've also worked on autoschedulers that use cost models or machine learning to generate target-specific schedules automatically. These work well for common patterns but still require manual tuning for peak performance in many cases.

Kara Rousseau Where else have domain-specific languages shown similar benefits? Is image processing unique, or does this approach generalize?

Dr. Jonathan Ragan-Kelley Image processing is particularly well-suited because it has regular data access patterns and abundant parallelism, but the approach generalizes to other domains with similar structure. Array programming languages for numerical computation, tensor languages for machine learning, query languages for databases—all exploit domain structure for optimization. TVM for machine learning compilation, for instance, uses ideas similar to Halide to separate operator implementation from scheduling strategy. Spatial languages for hardware synthesis let you describe dataflow architectures at a high level and then map them to FPGAs or custom ASICs through scheduling. The common thread is domains where you can identify a small set of fundamental operations and composition patterns, then provide a restricted language for those patterns. The restrictions enable powerful optimization that would be impractical if you had to handle arbitrary programs. The challenge is identifying domains with enough regularity and commercial value to justify building a specialized language and compiler infrastructure.

Sam Dietrich What about the learning curve? Do programmers find it easier to work in a DSL once they understand the domain model, or does the restriction feel limiting?

Dr. Jonathan Ragan-Kelley It depends on background. For researchers and engineers already working in the domain, DSLs typically feel natural because they align with how experts think about problems. An image processing researcher thinks in terms of stencils, reductions, and pipelines. Halide makes that mental model explicit rather than forcing translation to imperative loops and pointers. The learning curve is primarily in understanding the scheduling model—what transformations are available and how they affect performance. For generalists coming from C++ or Python, there's adjustment. The restriction to pure functions feels unnatural if you're used to imperative programming with mutable state. The need to think about scheduling separately from algorithm is a new concept. But most people find that once they internalize the model, productivity increases significantly. You think about the problem at a higher level and the compiler handles low-level details. The limitation comes when you hit the boundaries of what the DSL can express. Then you must either work around the restriction awkwardly or drop down to a general-purpose language for that piece.

Kara Rousseau Could we imagine a spectrum of DSLs with varying levels of specificity, or does each domain require a custom language?

Dr. Jonathan Ragan-Kelley There's definitely a spectrum. At one end, you have extremely specific languages for narrow domains—a DSL for raytracing, or for finite element analysis. These can be highly optimized but apply to very limited problems. At the other end, you have extensible frameworks that let you define domain-specific constructs within a general-purpose language, like embedded DSLs in Haskell or Scala. Halide is somewhere in the middle—specific enough to enable strong optimization but general enough to cover a broad class of image processing tasks. There's also a question of composability. Can you embed one DSL within another, or interface multiple DSLs within a program? This is an active research area. Ideally, you'd have a modular approach where you compose domain-specific components, each optimized for its domain, without losing the benefits of specialization. But this requires careful design of interface boundaries and agreement on data representations.

Sam Dietrich How do DSLs interact with hardware evolution? Do they become obsolete as architectures change, or do they adapt more easily than general-purpose code?

Dr. Jonathan Ragan-Kelley In principle, DSLs should adapt more easily because the separation of algorithm and schedule means you only need to change scheduling strategies, not the core algorithm. In practice, it depends on how well the DSL's abstraction aligns with new hardware capabilities. When GPUs added tensor cores for matrix multiplication, libraries for deep learning could exploit them relatively easily by adding new schedules. But if hardware introduces fundamentally new computation models that don't fit the DSL's abstraction—say, probabilistic computing or dataflow architectures—you might need to extend or redesign the DSL. The advantage over general-purpose code is that the changes are localized to the compiler and scheduling infrastructure rather than scattered throughout application code. You can maintain backward compatibility at the algorithm level while generating different code for new hardware. This has been valuable in practice as we've moved from scalar CPUs to SIMD, to GPUs, to heterogeneous systems with specialized accelerators.

Kara Rousseau What are the biggest open challenges in DSL design and optimization?

Dr. Jonathan Ragan-Kelley One is autoscheduling—automatically finding good optimization strategies without manual tuning. Current approaches use cost models or learned models, but they struggle with complex interaction effects between transformations. The schedule search space is enormous and non-convex. Another challenge is composability—how do you interface different DSLs or embed DSLs within general-purpose programs without losing optimization opportunities? Cross-domain fusion could be valuable but requires agreeing on data representations and coordination between compilers. There's also the question of verifying correctness of DSL compilers. The complexity of modern optimizing compilers makes bugs inevitable. Formal verification of DSL compilers is more tractable than for general-purpose compilers because the input language is restricted, but it's still challenging. Finally, there's the social challenge of adoption. Building a high-quality DSL and compiler requires significant engineering effort. Unless there's a large enough community or commercial demand, it's difficult to sustain.

Sam Dietrich Looking ahead, do you see DSLs becoming more prevalent, or will improvements in general-purpose compiler technology make them less necessary?

Dr. Jonathan Ragan-Kelley I think DSLs will become more prevalent, but in specific domains rather than replacing general-purpose languages. As hardware becomes more heterogeneous and specialized, the gap between high-level code and efficient execution widens. General-purpose compilers struggle because they can't assume much about program structure. DSLs bridge this gap by encoding domain knowledge. We're already seeing this in machine learning with frameworks like TensorFlow and PyTorch, which are essentially DSLs for tensor computation. In scientific computing, languages like Julia enable domain-specific optimization through multiple dispatch and type specialization. The pattern is that domains with significant commercial or scientific value develop specialized languages to exploit their structure. But most application code will remain in general-purpose languages because the problems don't have enough regularity to justify a DSL. The future is likely hybrid—general-purpose languages for control flow and orchestration, DSLs for performance-critical computational kernels, with carefully designed interfaces between them.

Kara Rousseau Dr. Ragan-Kelley, thank you for this exploration of domain-specific languages and the trade-offs between generality and performance.

Dr. Jonathan Ragan-Kelley Thank you. I enjoyed the conversation.

Sam Dietrich That's our program. Until tomorrow, remember that the right abstraction depends on the problem you're solving.

Kara Rousseau And that constraints can enable rather than limit. Good night.