Applying Tom Ngo's Embedded Constraint Graphics to Direct-Manipulation Breakfast Selection (Direct manipulation over simplicial complexes using barycentric interpolation: they're not just for breakfast any more.)

The Breakfast Simplex is a space of recipes parameterized by {egg, milk, flour} ratios, normalized onto a simplex. Add butter or sugar and the dimension increases. Add prep method and you create adjacent regions. A breakfast buffet is a larger, possibly disconnected simplicial complex spanning multiple ingredient families.

That structure is exactly what Tom Ngo formalized and patented in Embedded Constraint Graphics in 1996 at Interval Research Corporation. I wrote about it when the patent expired in 2016:

https://news.ycombinator.com/item?id=12572696

US Patent #5933150: System for image manipulation and animation using embedded constraint graphics

https://patents.google.com/patent/US5933150

When I asked Tom about applying ECG to other applications after the patent expired, he wrote:

>I am, of course, partial to the idea that gluing high-dimensional simplices at their edges and faces is an extremely general way to represent blending manifolds, in the same way that gluing polygons together has done us so much good in the 3D modeling space. I also think the >2 decades of progress since ECG have put us in a better position to do something really cool based on direct manipulation.

Golan Levin, Malcolm Slaney, and Tom Ngo used the ECG graphical editor to build the vector face cartoons for Mouther, simply by dragging eyes, mouths, and features directly on the drawing:

https://web.archive.org/web/20180717222910/http://www.flong....

ECG defines example states at vertices. Compatible examples span simplices. The full state space is a simplicial complex. Interior points are barycentric blends.

https://en.wikipedia.org/wiki/Simplicial_complex

https://en.wikipedia.org/wiki/Barycentric_coordinate_system

When you drag something in screen space, the system maps that motion into the n-dimensional interpolation space and solves for blend weights via the Moore–Penrose pseudoinverse of the Jacobian matrix, the same linear algebra used in inverse kinematics.

https://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_inverse

https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinan...

https://en.wikipedia.org/wiki/Inverse_kinematics

You don’t indirectly adjust abstract sliders. You directly manipulate concrete outcomes. The solver recovers coordinates.

The same formulation applies to interpolating vector drawings, mesh blending, facial animation, pose spaces, and other example-based interfaces where states are meaningful and compatibility matters.

The same geometric intuition appears in large language models. Tokens and concepts are represented as high-dimensional vectors, and model activations are computed through weighted linear combinations in embedding space. Interpolating between embeddings corresponds to moving through that vector space via weighted blends, just in many more dimensions. ECG makes the simplices explicit and topologically structured, while LLM representations are implicit and learned. In both cases, behavior emerges from interpolation in high-dimensional spaces.

Apple’s ARKit already exposes facial expression as a set of named blend shape coefficients via ARFaceAnchor — values like mouthSmileLeft, jawOpen, and eyeBlinkRight driving a 3D face mesh in real time. Bring Mouther into 3D and you can drag the mouth corners upward to interpolate toward smiling targets, mapping that motion through the same barycentric machinery into blend weights instead of hard-coded sliders. This would make a great Blender plug-in for directly manipulating facial animation, to use with FaceIt!

Faceit : Facial Expressions And Performance Capture

https://superhivemarket.com/products/faceit

Breakfast is a concrete instance. Pancake, crepe, and omelette define a simplex over ingredient ratios. Drag toward eggs and the egg weight increases. Drag toward milk and you move along that axis. Cross the egg-milk edge shared by the crepe simplex {flour, egg, milk} and the custard simplex {egg, milk, sugar}, and you move from thin batters into sweet custards without leaving the manifold. The Dark Breakfast region is simply an unoccupied part of a valid simplex -- suggesting adjacent, unexplored Dark Custard subspaces rather than forbidden states.

Simplicial complexes are useful UI primitives. They provide local linear interpolation inside zones and explicit global topology across zones. They scale to higher dimensions, while maintaining a user friendly 2D direct manipulation user interface. They encode constraints structurally instead of procedurally.

A pie menu can be viewed as a radial parameterization of a simplex. A direct-manipulation pie menu over ingredient space lets you drag in the direction of the crusts and fillings you want, with barycentric weights accumulating as you move.

The Design and Implementation of Pie Menus (Dr. Dobb’s Journal, Dec. 1991, cover story, user interface issue.)

https://donhopkins.medium.com/the-design-and-implementation-...

An Empirical Comparison of Pie vs. Linear Menus (Jack Callahan, Don Hopkins, Mark Weiser and Ben Shneiderman. Presented at ACM CHI’88 Conference, Washington DC, 1988.)

https://donhopkins.medium.com/an-empirical-comparison-of-pie...

The Hunt for Dark Breakfast