CELLNET 6-1.gif

CELLNET Interface 1.0

Node-based adaptive cellular network GUI for interactive environments(Ongoing Project)

Personal Thesis Project 

Master of Science in Matter Design Computation(M.S.MDC), Cornell AAP

Working period : Jan.2022 ~ May.2022(still ongoing after the semester)

 

AAP-College-External-black.png

How might we control the holographically generated cellular system or interface by external stimuli and see its interplayable adaptiveness with actual architectural environment? 

 

Started with this initial question, in this ongoing thesis project—CellNet Interface—in Cornell M.S.MDC program, I conducted tangible user interface(TUI)-centered research and simulation on how hand gesture or haptic behavioral pattern could manipulate the the system of bio-inspired computated cellular interface and system. Integrated with nature's microscopic cellular systems like arabidopsis' sepal cell structure, in overall sequences between usability-experimentation and simulation phases, this project experimentally validates the concepts of how primitve hand gestures that controls the cell structure's morphism could be patternized by UX-centered viewpoint, clarified into an archetype that ables to influence the parameters of computationally generated cellular system via AR/MR devices like leapmotion or Kinnect or Hololens. Through this trajectory, ultimately I intended to create the new type of mixed reality-based virtual interface that inherently contains and artificially shows microscopic bio-system's morphological, morphogenetic traits. 

Experiment Introduction:
Interactive Simulation via Leapmotion

To explain the overall simulation method, I utilized the Leapmotion(hand tracking device) as a main medium that senses user’s hand gestural movement and attempted to link the Leapmotion with the Grasshopper of Rhino3D. For developmental side, I developed the script, and utilized Kangaroo physics components in Grasshopper to technically allow the node-based structure to respond to user's hand gestures.

 

In initial stage of this project,  I gradually started to simulate on diverse types of bio-inspired cellular system like fruit fly drosophila’s embryo or sepal initiation’s cell structure, and which are also responsive to various hand gestural movements.

Screen Recording 2022-03-27 at 9.10.52 PM_1.gif
Screen Recording 2022-03-27 at 9.10.52 PM.gif

 Figure 1: Scenes of simulating bio-inspired, node-based cellular systems by hand tracking technology via Leapmotion

(Left): Cell Network inspired by Fruit fly drosophila’s embryo

(Right): Cell Network inspired by Sepal Initiation’s cell structure

Technical Direction Point

As simply explained right above, I utilized Leapmotion as an interactive medium that senses diverse types of my hand gestural movement, and through this initial phase, my hand’s physical movement, velocity, positional datas are translated into analogue numerical values, then scripted in the grasshopper modeling. Though this process, those kinds of translated values are used as parameters that control the structural, morphological characteristics of digitized cellular system like phase 3 in the diagram.

Technical Direction Point.png

 Figure 2: A procedural flow that explains how user's hand gesture could be interpreted, translated into parameters that control the bio-inspired cellular system within this project

Sequential Project Process

In Section 1 covering phase 0 to 1, most of tasks are based on research and initial concept modeling. Especially on field research part on phase 0.5, I deep-dived into biology field to find the linkage to my thesis direction point, and had several individual sessions with biology PhD student to discuss with crossdisciplinary point of views over my thesis project.

Then in Section 2, based on field research and initial concept modeling and development, I conduct several types of experimentation and simulation on manipulating morphological, morphogenetic cellular system traits like manipulating cell wall stiffness, spike length, density via changing haptic gestures.

 

And in phase 3, based on those simulation, I conduct quantitative, qualitative analysis to clarify the artificial rule system analogous to nature’s one, and simultaneously also design UX principles about controlling 3 dimensional holographic models with physical stimuli like hand gestural movements.

 

And lastly in phase 4, I upscale the digitized cellular system to an interior structural size, and develop it adaptable to an actual physical environment via AR/MR devices like leap motion, Kinect and Hololens.

Usability Definition_ Haptic Gesture.jpg

 Figure 3: A flow chart for an entire project planning 

Chapter 1:
Digitizing cell structure and simulating its morphological traits

Procedural Process -
From bio-inspired structure to node network-based cell system

In an initial stage, inspired by nature’s microscopic structural characteristics like organic shape, sepal structure, fruit fly drosophila embryo and so on, I extracted structural traits from them and tried to apply those structures to different types of node network and mesh structural modelings to see how the bio-inspired structure could be translated into digitized node-based cell structures with divesely differentiated conditions.

Future Scenario Rendering2.jpg

 Figure 4: A procedural diagram that shows how micro-nature's cellular systems could be digitally translated into diverse types of node-based artificial cellular systems and computationally reorgarnized into procedural processes

Simulation 1 :
Morphogenetic response towards directional mechanical stress

In simulation 1, through developing the script via Kangaroo physics, Leapmotion, and Leapmotion bridge component within Grasshopper, I simulated cellular system’s morphogenetic respsones towards directional mechanical stress in terms of biophysics. Presetted with assigning an attractor point on to my index finger when it comes to sensing my hand gesture via Leapmotion, a simulation has been tested and I tried to clarify the certain parameter conditions along with calculating an attractor point’s vector velocity.

Precedent Study2.jpg

 Figure 5: A diagram of visually explaining node-based cellular system's physic reaction towards directional mechanical stress along with clarifying vector velocity

Simulation 2 :
Morphogenetic response towards several internal rules

As a continuous analytic process on Simulation 1, in this stage I attempted to simulate different types of cell structure under specified conditions of gradually manipulating certain parameters like spring stiffness on cell network—which is cell membrane, or spring damping—which is cell network resilience, or tri-mesh iteration—which is cell nuclei population.

Future Scenario Rendering3.jpg

 Figure 6: Diagrams of explaining procedural variations for different types of node-based cellular systems under certain conditions(e.g. increasing spring stiffness, damping, trimesh iteration)

Particularly in the second simulation of sepal initiation shape as multiple cellular form with curved shape, the structure in changing spring stiffness condition showed an interesting result, which is that as you can see the highlighted area on top right, the center area of sepal shape shows more dynamic reaction towards decreasing spring stiffness. This became a useful insight for me to model a multiple cellular form-based organic shape.

Comparison graph between
different cell structure types:
1) Spring Stiffness

 

Based on simulation with situating different parameter conditions, I analyzed different types of cellular systems into this consulted diagram. First of all, in this diagram that measures spring stiffness on cellular forms, a singular cellular form shows rapid decrease in gradually increasing spring stiffness whereas a multi-cellular form is relatively more advantageous when it comes to resisting towards spring stiffness(e.g. Cell wall stiffness, regulatory hormone system that inhibits cell’s morphogenesis).

 

> For singular cellular form, its formative activity could be significantly declined by increasing spring stiffness

Cell Diagram 1.png

 Figure 7: A diagram of cell structure's structural variation on spring stiffness change

Comparison graph between
different cell structure types:

2) Spring Damping

In analysis on cell structure’s structural variation towards spring damping-which is cell structural resilience, a multi-cellular form with curved shape shows rapid decrease in gradually increasing spring damping. This means Multi-cellular form(more than 4~5 units) with different inner shapes is relatively more disadvantageous when it comes to resisting towards spring damping in terms of cell resilience.

Cell Diagram 2.png

 Figure 8: A diagram of cell structure's structural variation on spring damping change

Comparison graph between
different cell structure types:

3) Total amount of cell vertices(cell nuclei)

And lastly, in analysis on cell structure’s structural variation towards cell nuclei’s total amount change, Multi-cellular forms show a slightly steeper increase in increasing tri-mesh iteration than singular one. Which means that multi-cellular form is relatively more advantageous for cellular system’s morphogenesis.

Cell Diagram 3.png

 Figure 9: A diagram of cell structure's structural variation on changing total amount of cell vertices 

Chapter 2:
Simulating cellular form with multiple hand gestures 

Sensor-driven cellular form's biomimetic response simulation
via mutiple hand gestures 

In this stage, based on previous simulations I have been conducting regarding cell structure’s morphogenetic, morphological change, I gradually started to actively utilize the multi-typed hand gestural movements to reactive cellular system and tried to see the result by synthesizing with artificial system like spike structure. The diagram below visually explains how this spike system-embedded cell structure could be responsively reacted to user’s multiple hand gestures in realtime via leapmotion.

Precedent Study3.jpg

 Figure 10: A diagram that explains the sequential process of how multiple hand gestures could be tranlated, converted into biomimetic movement of cellular forms

Designing basic UX principle on hand gestural movments with Leapmotion

In designing basic UX Principle on hand gestural movement with Leapmotion, I established several types of hand gestural characteristics that could be potentially used in cell structure’s bio-mimetic response simulation. Through these UX archetypes, I was able to significantly clarify the key UX points while defining skeletal movement of my hand and finger’s physical movement like velocity or x,y,z coordinate-based positional change.

Thesis Planning.png

 Figure 11: A diagram of explaining several types of hand gestures linked with Leapmotion

Simulating cellular system’s morphological reaction towards hand gestural movement 

The simulations below are indicating differently assigned conditions to spike surface-integrated cell structures along with differentiating multiple hand gestures. As the result, by differentiating combination of multiple hand gestures like hovering, rotating or adjusting the distance between a hand and leap motion sensor, the spike system shows highly responsive, organic-like movement that resembles nature’s coral plant.

Default Condition

- Cell Wall Stiffness : 31.79

- Cell Wall Damping :  22.16

- Distance between Sepal and Fingertip : Variable

  (Realtime sensing via Leapmotion)

ezgif.com-gif-maker (3).gif
Screen Shot 2022-06-13 at 6.09_edited.pn

 Figure 12: A round tip spike tracing and responding to a hand gesture hovering and going up and down

ezgif.com-gif-maker (5).gif
Screen Shot 2022-06-13 at 6.09_edited.pn

 Figure 13: Expanding and shrinking round tip spike by adjusting distance between hand and Leapmotion

ezgif.com-gif-maker (4).gif
Screen Shot 2022-06-13 at 6.09_edited.pn

 Figure 14: A round tip spike with flourescent color texture tracing and responding to a hovering hand 

ezgif.com-gif-maker (10).gif
Screen Shot 2022-06-13 at 6.09_edited.pn

 Figure 15: A spreading spike via clenching, opening fist and pinching motion

ezgif.com-gif-maker (11).gif
Screen Shot 2022-06-13 at 6.09_edited.pn

 Figure 16: A spreading spike with flourescent color texture via clenching, opening fist and pinching motion 

Computational Synthesis between cellular system and vasculature system’s microtubulus

 In this simulation, I artificially integrated cellular structure with vasculature’s microtubules system which we can actually see in nature’s ecosystem, along with assigning spring force model in different conditions. What I was trying to see in this simulation was how cellular interface is interlinked via vasculature and its microtubules system could be working as a nutrient vessel analogous to our body’s or leaf’s vasculature system.

< Simulation condition >

 

- Each vasculature systems’ microtubulus should be linked with cell’s vertex so the vasculature could be responsively move towards cell system’s morphological change.

 

- Analogous to the previous exploration, vasculature and cell system reacts to spring force’s vertical movement that symbolizes the hand gestural movement.

Screen Shot 2022-04-18 at 8.36.42 AM.png

 Figure 17: A node-based cellular form of which vertices are linked with microtubulus network 

Sepal2.gif

 Figure 18: A vasculature system-integrated cellular form of which the diameter of its microtubulus and the thickness of cell wall are changing at the same time

Sepal3.gif

 Figure 19: A second iteration on vasculature system-integrated cellular form of which the diameter of its microtubulus is changing and the thickness of cell wall increases and decreases outward on Z axis direction

Sepal4.gif

 Figure 20: A simulation on cell wall stiffness linked with anchor vertex's Z-axis directional movement that controls cell vertices' physic reaction

Sepal7.gif

 Figure 21: A vasculature system-integrated cellular form of which the diameter of its microtubulus and the thickness of cell wall are changing according to anchor vertex's Z-axis directional movement

Sepal10.gif

 Figure 21: A vasculature system-integrated fruit drosophila structure of which the diameter of its microtubulus and the thickness of cell wall are changing according to changing spring stiffness

Sepal9.gif

 Figure 22: A vasculature system-integrated sepal initiation structure of which the diameter of its microtubulus and the thickness of cell wall are changing according to changing spring stiffness 

Sepal8.gif

 Figure 23: A vasculature system-integrated flow primordia structure of which the diameter of its microtubulus and the thickness of cell wall are changing according to changing spring stiffness 

Simulation on Cellular System’s morphological reaction with certain conditions

As a further experimental attempt, I iterated another versions of simulating cell structures with situating diverse types of conditions: exaggerated vasculature system, regulatory system analogous to hormone system like Auxin and cytokinin, orienting vasculature system’s starting point via linked with finger position, adjusting spike’s length via two hand motion, controlling cell population via pinching hand gesture and generating vasculature system via pinching hand gesture.

ezgif.com-gif-maker (8).gif

 Figure 24: A vasculature system-integrated cellular form of which the diameters of microtubulus differently exaggerated analogous to human organ like kidney, responding to pinching hand motion via Leapmotion

ezgif.com-gif-maker (9).gif

 Figure 25: A simulation on regional regulatory system inspired by biomicroscopic regulatory system found in plant organism: Auxin(the hormone that promotes cell growth) and Cytokinin(the hormone that inhibits cell growth)

ezgif.com-gif-maker (6).gif

 Figure 26: A simulation on manipulating orient position of vasculature network via tracking the finger tip point's XYZ coordinates by Leapmotion

ezgif.com-gif-maker (12).gif

 Figure 27: A simulation on responsively spreading out spikes by two-hand gestural motion

ezgif.com-gif-maker (13).gif

 Figure 28: A simulation on controlling cell vertices population by pinching hand gesture

ezgif.com-gif-maker (7).gif

 Figure 29: A simulation on generating vasculature network by pinching hand gesture 

Speculative rendering for the next project trajectory:
Adaptive cellular structure to actual environment 

Expanding responsive node-based cellular structure to actual environment

Based on responsive, biomimetic cellular forms I simulated in former phase, I talked with Jenny(Program Director), gradually starting to envisioning about creating an architectural cellular structure reciprocally adaptive to the characteristics of physical environments. Analogous to morphogen gradient or reactions that we observe in microscopic nature, initially I intended to design a haptic responsive, cellular structure-based environmental interface generated via Microsoft Hololens and it reciprocally coexists with physical surroundings by Hololens' rendering technology. This part will be deeply discussed with Jenny and the other thesis advisors and developed as a working mockup by utilizing Hololens during next semester but as a speculative direction point I modeled the architectural cellular structures via Grasshopper and Rhino 3D, and merged them with the actual environment via Photoshop to speculatively show how the next movement of my thesis activity would be. 

Screen Shot 2022-05-05 at 3.58.57 PM.png

 Figure 30: A footage of modeling a cellular form with round tip spike system via Grasshopper and Rhino 3D

Cell Environment 2.png

 Figure 31: A rendering shot of virtually touching the responsive cellular structure coexisting with the actual environment(actual venue: Cornell Milstein Hall) 

Cell Environment 1.png

 Figure 32: A modeling for the architectural cellular structure 

Final Render 1.png

 Figure 33: A final rendering shot that explains how the user manipulates the parameters of architectural cellular structure that coexists with physical environment via augmented interface within Hololens experience (actual venue: Cornell Milstein Hall)

 Figure 34: A compilation video of simulation tests on hand gesture-responsive cellular structures

CELLNET Interface 1.0

Node-based adaptive cellular network GUI for interactive environments

(Ongoing Project)

Personal Thesis Project 

Master of Science in Matter Design Computation(M.S.MDC), Cornell AAP

Working period : Jan.2022 ~ May.2022

 

AAP-College-External-black.png