CELLNET
Bio-inspired artificial cellular structure adaptable to physical stimuli and an environment
Personal Thesis Project
Master of Science in Matter Design Computation(M.S.MDC), Cornell AAP
Working period : Jan.2022 ~ (Ongoing project)
How might bio-inspired cellular user interface design adapt to physical stimuli and an environment?
In witnessing the evolution of how tangible user interface(TUI) integrated with emerging technology interplays with users in diverse forms of platforms, its user experience has been also correspondingly evolved, adaptively redefined into the system of which the interface is designed upon physical environment as a superimposed way . In this project, CellNet demonstrates several experimental possibilities of how microscopic cellular system-inspired graphic user interface(GUI) could digitally replicate cellular traits like morphogenetic movements or collective cell division, and be manipulated by external stimuli and adjacent environments. In audience aspect, a group of Milennials and Generation Z(aging from 18 ~ 40 at this moment of 2022) is targeted as a main user familiar with AR/VR-integrated experiential life style. In the process of designing interface that interplays with external stimuli, node-based cellular network interface is mainly considered, prototyped with parameterized variables. Also, this project explores UX-driven gestural principles on how the user manipulates the node properties of network interface by diverse types of haptic gestures and physical object movements. Combined with these two major trajectories, this thesis explores the adjacent opportunity on how the usability of this bio-inspired node network interface facilitates to user adaptivity of perceiving augmented mixed reality to a proper degree.
Keywords
Tangible User Interface(TUI); Graphic User Interface(GUI); User Experience(UX); Parametric Modeling; Gestural Pattern
Activities of simulating diverse types of nature-inspired artificial cell structures in CellNet projects
Experiment Introduction:
Interactive Simulation via Leapmotion
To explain the overall simulation method, I utilized the Leapmotion(hand tracking device) as a main medium that senses user’s hand gestural movement and attempted to link the Leapmotion with the Grasshopper of Rhino3D. For developmental side, I developed the script, and utilized Kangaroo physics components in Grasshopper to technically allow the node-based structure to respond to user's hand gestures.
In a very initial stage of this project, I gradually started to model and simulate bio-inspired cellular system like fruit fly drosophila’s embryo or sepal initiation’s cell structure, and which are also responsive to various hand gestural movements.
Figure 1: Scenes of simulating bio-inspired, node-based cellular systems by hand tracking technology via Leapmotion
(Left): Cell Network inspired by Fruit fly drosophila’s embryo
(Right): Cell Network inspired by Sepal Initiation’s cell structure
Thesis Concept Logic
In utilizing Leapmotion as a sensor that detects hand gestural(mostly) movements, I established several principles that embody how the sensed analogue values are computationally converted into design drivers that influence artificial cell structure's morphogenetic morphological movements. Based upon tracking user’s physical gestural patterns, those external stimuli are translated into skeletal points and frames, then analyzed into several variables like XYZ coordinates, distance value between object and sensor, proximity value between two index fingers, and velocity of gestural movment and so on. And those parametric variables are applied to computationally rendered cellular structure, and facilitate to make it movable in morphogenetic, morphological ways, and also responsive to physical gestures.
Figure 2: A diagram for transitional relationship between physical stimuli and artificial cell structure's morphological reaction. Between those two aspects, sensed analogue values are converted into artificial cell's morphological, morphogenetic movements.
Figure 3: A variety of artificial cellular systems’ morphogenetic movements in this project. Derived from microscopic nature’s morphogenetic movements, CellNet comprehensively investigated these 10 types of traits which could be computationally rendered in modeling: Cell elongation, Spreading directionality, Cell noise, Incremental cell growth, Cell mesh conversion, Controlling cell tip sharpness, Spring stiffness, cell wall thickness, cell division, different regulatory system.
Sequential Project Process
In Section 1 covering phase 0 to 1, most of tasks are based on research and initial concept modeling. Especially on field research part on phase 0.5, I deep-dived into biology field to find the linkage to my thesis direction point, and had several individual sessions with biology PhD student to discuss with crossdisciplinary point of views over my thesis project.
Then in Section 2, based on field research and initial concept modeling and development, I conduct several types of experimentation and simulation on manipulating morphological, morphogenetic cellular system traits like manipulating cell wall stiffness, spike length, density via changing haptic gestures.
And in phase 3, based on those simulation, I conduct quantitative, qualitative analysis to clarify the artificial rule system analogous to nature’s one, and simultaneously also design UX principles about controlling 3 dimensional holographic models with physical stimuli like hand gestural movements.
And lastly in phase 4, I upscale the digitized cellular system to an interior structural size, and develop it adaptable to an actual physical environment via AR/MR devices like leap motion, Kinect and Hololens.
Figure 4: A flow chart for an entire project planning
Figure 5: Overall information architecture-based algorithmic flow that explains the basic procedural flow of how the user interacts with the interface and reaches to cellular system on the end point. Starting with recognziing the actuator like Hololens that allows users to enter the Mixed Reality environment, users go through sequential processes of interim graphic user interfaces in terms of information architecture hierarchy as showing in the middle area and eventually interacts with artificial cellular system on the end point.
Chapter 1:
Digitizing cell structure and simulating its morphological traits
Procedural Process -
From bio-inspired structure to node network-based cell system
In an initial stage, inspired by nature’s microscopic structural characteristics like organic shape, sepal structure, fruit fly drosophila embryo and so on, I extracted structural traits from them and tried to apply those structures to different types of node network and mesh structural modelings to see how the bio-inspired structure could be translated into digitized node-based cell structures with divesely differentiated conditions.
Figure 6: A procedural diagram that shows how micro-nature's cellular systems could be digitally translated into diverse types of node-based artificial cellular systems and computationally reorgarnized into procedural processes
Simulation 1 :
Morphogenetic response towards directional mechanical stress
In simulation 1, through developing the script via Kangaroo physics, Leapmotion, and Leapmotion bridge component within Grasshopper, I simulated cellular system’s morphogenetic respsones towards directional mechanical stress in terms of biophysics. Presetted with assigning an attractor point on to my index finger when it comes to sensing my hand gesture via Leapmotion, a simulation has been tested and I tried to clarify the certain parameter conditions along with calculating an attractor point’s vector velocity.
Figure 7: A diagram of visually explaining node-based cellular system's physic reaction towards directional mechanical stress along with clarifying vector velocity
Simulation 2 :
Morphogenetic response towards several internal rules
As a continuous analytic process on Simulation 1, in this stage I attempted to simulate different types of cell structure under specified conditions of gradually manipulating certain parameters like spring stiffness on cell network—which is cell membrane, or spring damping—which is cell network resilience, or tri-mesh iteration—which is cell nuclei population.
Figure 8: Diagrams of explaining procedural variations for different types of node-based cellular systems under certain conditions(e.g. increasing spring stiffness, damping, trimesh iteration)
Particularly in the second simulation of sepal initiation shape as multiple cellular form with curved shape, the structure in changing spring stiffness condition showed an interesting result, which is that as you can see the highlighted area on top right, the center area of sepal shape shows more dynamic reaction towards decreasing spring stiffness. This became a useful insight for me to model a multiple cellular form-based organic shape.
Comparison graph between
different cell structure types:
1) Spring Stiffness
Based on simulation with situating different parameter conditions, I analyzed different types of cellular systems into this consulted diagram. First of all, in this diagram that measures spring stiffness on cellular forms, a singular cellular form shows rapid decrease in gradually increasing spring stiffness whereas a multi-cellular form is relatively more advantageous when it comes to resisting towards spring stiffness(e.g. Cell wall stiffness, regulatory hormone system that inhibits cell’s morphogenesis).
> For singular cellular form, its formative activity could be significantly declined by increasing spring stiffness
Figure 9: A diagram of cell structure's structural variation on spring stiffness change
Comparison graph between
different cell structure types:
2) Spring Damping
In analysis on cell structure’s structural variation towards spring damping-which is cell structural resilience, a multi-cellular form with curved shape shows rapid decrease in gradually increasing spring damping. This means Multi-cellular form(more than 4~5 units) with different inner shapes is relatively more disadvantageous when it comes to resisting towards spring damping in terms of cell resilience.
Figure 10: A diagram of cell structure's structural variation on spring damping change
Comparison graph between
different cell structure types:
3) Total amount of cell vertices(cell nuclei)
And lastly, in analysis on cell structure’s structural variation towards cell nuclei’s total amount change, Multi-cellular forms show a slightly steeper increase in increasing tri-mesh iteration than singular one. Which means that multi-cellular form is relatively more advantageous for cellular system’s morphogenesis.
Figure 11: A diagram of cell structure's structural variation on changing total amount of cell vertices
Chapter 2:
Simulating cellular form with multiple hand gestures
Sensor-driven cellular form's biomimetic response simulation
via mutiple hand gestures
In this stage, based on previous simulations I have been conducting regarding cell structure’s morphogenetic, morphological change, I gradually started to actively utilize the multi-typed hand gestural movements to reactive cellular system and tried to see the result by synthesizing with artificial system like spike structure. The diagram below visually explains how this spike system-embedded cell structure could be responsively reacted to user’s multiple hand gestures in realtime via leapmotion.
Figure 12: A diagram that explains the sequential process of how multiple hand gestures could be tranlated, converted into biomimetic movement of cellular forms
Designing basic UX principle on hand gestural movments with Leapmotion
In designing basic UX Principle on hand gestural movement with Leapmotion, I established several types of hand gestural characteristics that could be potentially used in cell structure’s bio-mimetic response simulation. Through these UX archetypes, I was able to significantly clarify the key UX points while defining skeletal movement of my hand and finger’s physical movement like velocity or x,y,z coordinate-based positional change.
Figure 13: A diagram of explaining several types of hand gestures linked with Leapmotion
Simulating cellular system’s morphological reaction towards hand gestural movement
The simulations below are indicating differently assigned conditions to spike surface-integrated cell structures along with differentiating multiple hand gestures. As the result, by differentiating combination of multiple hand gestures like hovering, rotating or adjusting the distance between a hand and leap motion sensor, the spike system shows highly responsive, organic-like movement that resembles nature’s coral plant.
Default Condition
- Cell Wall Stiffness : 31.79
- Cell Wall Damping : 22.16
- Distance between Sepal and Fingertip : Variable
(Realtime sensing via Leapmotion)
Figure 14: A round tip spike tracing and responding to a hand gesture hovering and going up and down
Figure 15: Expanding and shrinking round tip spike by adjusting distance between hand and Leapmotion
Figure 16: A round tip spike with flourescent color texture tracing and responding to a hovering hand
Figure 17: A spreading spike via clenching, opening fist and pinching motion
Figure 18: A spreading spike with flourescent color texture via clenching, opening fist and pinching motion
Computational Synthesis between cellular system and vasculature system’s microtubulus
In this simulation, I artificially integrated cellular structure with vasculature’s microtubules system which we can actually see in nature’s ecosystem, along with assigning spring force model in different conditions. What I was trying to see in this simulation was how cellular interface is interlinked via vasculature and its microtubules system could be working as a nutrient vessel analogous to our body’s or leaf’s vasculature system.
< Simulation condition >
- Each vasculature systems’ microtubulus should be linked with cell’s vertex so the vasculature could be responsively move towards cell system’s morphological change.
- Analogous to the previous exploration, vasculature and cell system reacts to spring force’s vertical movement that symbolizes the hand gestural movement.
Figure 19: A node-based cellular form of which vertices are linked with microtubulus network
Figure 20: A vasculature system-integrated cellular form of which the diameter of its microtubulus and the thickness of cell wall are changing at the same time
Figure 21: A second iteration on vasculature system-integrated cellular form of which the diameter of its microtubulus is changing and the thickness of cell wall increases and decreases outward on Z axis direction
Figure 22: A simulation on cell wall stiffness linked with anchor vertex's Z-axis directional movement that controls cell vertices' physic reaction
Figure 23: A vasculature system-integrated cellular form of which the diameter of its microtubulus and the thickness of cell wall are changing according to anchor vertex's Z-axis directional movement
Figure 24: A vasculature system-integrated fruit drosophila structure of which the diameter of its microtubulus and the thickness of cell wall are changing according to changing spring stiffness
Figure 25: A vasculature system-integrated sepal initiation structure of which the diameter of its microtubulus and the thickness of cell wall are changing according to changing spring stiffness
Figure 26: A vasculature system-integrated flow primordia structure of which the diameter of its microtubulus and the thickness of cell wall are changing according to changing spring stiffness
Simulation on Cellular System’s morphological reaction with certain conditions
As a further experimental attempt, I iterated another versions of simulating cell structures with situating diverse types of conditions: exaggerated vasculature system, regulatory system analogous to hormone system like Auxin and cytokinin, orienting vasculature system’s starting point via linked with finger position, adjusting spike’s length via two hand motion, controlling cell population via pinching hand gesture and generating vasculature system via pinching hand gesture.
Figure 27: A vasculature system-integrated cellular form of which the diameters of microtubulus differently exaggerated analogous to human organ like kidney, responding to pinching hand motion via Leapmotion
Figure 28: A simulation on regional regulatory system inspired by biomicroscopic regulatory system found in plant organism: Auxin(the hormone that promotes cell growth) and Cytokinin(the hormone that inhibits cell growth)
Figure 29: A simulation on manipulating orient position of vasculature network via tracking the finger tip point's XYZ coordinates by Leapmotion
Figure 30: A simulation on responsively spreading out spikes by two-hand gestural motion
Figure 31: A simulation on controlling cell vertices population by pinching hand gesture
Figure 32: A simulation on generating vasculature network by pinching hand gesture
Simulating regional regulatory system inspired by plant’s hormone signaling and regulatory system
As a main design driver for simulation, inspired by plant cell’s hormone signaling and regulatory systems that regulates cell growth like Auxin, Cytokinin or Gibberellin, I computationally scripted several types of cell structures of which morphogenetic movements are influenced by regulatory rules. For instance, I scripted two distinguished areas like the one for cell growth promoting area whereas the other area for cell grwoth inhibiting area inspired by plant cell's hormone regulatory system. In sequentially developing the simulations regarding cell growth regulatory system, a cell structure was developed into another model responsive to hand gesture, also converted into a downscaled model in architectural case studies.
Figure 33: As a basic approach, I computationally scripted “Morphogenetic cell growth emergent”. As you can see the animation, cell is growing morphogenetic, volumetric way.
Figure 34: And then, based on a plant’s cell growth emergent regulatory rules, I scripted a cell growth inhibiting area(cell membrane), and the overall cell structure shows another morphogenetic reaction with extruding cells beyond the inhibiting cell membrane.
Figure 35: A multiple cell inhibiting areas-based simulation.
Figure 36: A diagram that explains conceptual transition from 2d to 3d simulation. Continuing hormone-inspried regulatory system, in 3d simulation along with sensing hand gestural movements, two areas for cell growth promotion and inhibition area are affected by the realtime value derived from hand gesture's distance value sensed between a hand and a Leapmotion sensor.
Figure 37: A footage of simulating two different types of cell structure(cell growth promoting area: right / cell growth inhibiting area: left) responsive to hand's vertical movement
Figure 38: This is a procedural workflow that explains how the two different types of cellular sytems are governed by two distinguishable plant hormone inspired regulatory rules. With scripting cell structure's procedural morphism responsive to hand's vertical distance value, I established and scripted the internal rules for cell geometry's volumetric elongation. Number was. “as the distance value is diminished, the cell structure grows by cell promoting hormone-inspired algorithm.” And for number 2, it was “as the distance value is increased, the cell structure growth is inhibited by cell growth inhibiting hormone inspired algorithm.”
Figure 39: A parametric result on responsive cell regulatory system-based cell structure. It is obviously showing that cell structure’s elongation degree is escalated in the end phase as the value of distance increases continuously.
Human-scaled cell structure superimposed with physical environment
Expanding responsive node-based cellular structure to actual environment
Based on responsive, biomimetic cellular forms I simulated in former phase, I talked with Jenny(Program Director), gradually starting to envisioning about creating an architectural cellular structure reciprocally adaptive to the characteristics of physical environments. Analogous to morphogen gradient or reactions that we observe in microscopic nature, initially I intended to design a haptic responsive, cellular structure-based environmental interface generated via Microsoft Hololens and it reciprocally coexists with physical surroundings by Hololens' rendering technology. Also via interim simulations I experientially accumulated, cell-growth regulatory system-driven cellular structures are developed, augmented, and upscaled into human-scale holograms superimposed with the physical environments.
Figure 40: Prior to simulating human-scaled hologrpahic simulation, I attempted several types of holographic modelings, simulations and also graphic renderings to speculate how the cell system could be immersively interactived with physical stimuli and environments.
*Top left: Simulating spike surface-based cell structure via Leapmotion and iPad’s Fologram interface *Top right: Simulating spike surface-based cell structure’s resiliency test via Leapmotion and Microsoft Hololens’ Fologram interface *Bottom left: Speculative render shot for human-scaled cell structure responsive to audience’s finger *Bottom right: Axonometric rendering shot of human-scale holographic cell structure that can be rendered by Hololens goggle
Figure 41: Axonometric Bird-eye view drawing of holographic cellular structure rendered via audience’s Hololens. In terms of considering spatial adaptability, I chose Cornell AAP's Milstein hall as an actual venue for simulation experience because I thought in a sense that especially the Milstein Hall’s dancing floor has a spacious area for a Hololens simulation, also contains proper amount of objects(e.g., desk, chairs, pillars…) so I can seek out the opportunities to adapt holographic cell structures to them, being able to see how the holograms interplays with not only environment but also the surrounding objects
Figure 42: A diagram that explains conceptual transition from simplified cell growth regulatory system-based simulation to human-scale simulation. Continuously keeping on track in cell growth regulatory system, again I expanded into architectural scale and of which morphological trait is influenced by architectural surroundings and conditions. For instance, in the picture on the right side, while the cell structure growth outward, blue area for a cell growth inhibiting area inhibits outgrowing cell structure.
Figure 43: Cell division simulation not affected by architectural surroundings - Phase 1
Figure 44: Cell outgrowth simulation affected by cell growth inhibiting area(blue area) - Phase 2
Figure 45: 3d Cell outgrowth simulation affected by cell growth inhibiting area(blue area) - Phase 3
Figure 46: 3d Another cell outgrowth simulation affected by cell growth inhibiting area - Cornell AAP's Milstein Dome area
Participatory User Testing Group Session with Microsoft Hololens and Kinect
(Ongoing)
For a wide simulation testing with a variety of participants having different propensity over mixed reality experience, I conducted a participatory user testing group session and usability survey on mixed reality with 2 groups of people. In proceeding user testing, two major aspects was dealt with when it comes to conducting MR experimentation via Hololens: Test 1 was an individual test about Usability and reactivity test on relatively small-sized holographic cell structure and Test 2 was a group test about Usability and reactivity test on human-scaled holographic cell structure.
Figure 47: *Top left: Propensity survey on Mixed Reality experience *Top right: Group 1’s user testing on human-scaled responsive cellular structure *Bottom left: Group 2’s user testing on humanscaled responsive cellular structure *Bottom right: Individual user testing on cellular structure’s cell elongation
* Compilation video on entire activities in CellNet project(user testing group session is included in the last part)
CELLNET
Bio-inspired artificial cellular structure
adaptable to physical stimuli and an environment
Personal Thesis Project
Master of Science in Matter Design Computation(M.S.MDC), Cornell AAP
Working period : Jan.2022 ~ May.2022
