Publications

We present an adaptation of the sigma-lognormal model to generate and fit smooth trajectories in conjunction with a differentiable vector graphics (DiffVG) rendering pipeline and with parameter selection driven by a minimum-time smoothing criterion. This approach enables the incorporation of the ``Kinematic Theory of Rapid Human Movements'' into modern image-based deep learning systems. We demonstrate its utility through various applications, including fitting handwriting trajectories to an image and generating trajectories using guidance from a large multimodal model.

@inproceedings{Berio25IGS,
  author = {Berio, D. and Calinon, S. and Plamondon, R. and Leymarie, F. F.},
  booktitle = {22nd Conference of the International Graphonomics Society ({{IGS}})},
  title = {Differentiable Rasterization of Minimum-Time Sigma-Lognormal Trajectories},
  url = {https://calinon.ch/papers/Berio-IGS2025.pdf},
  year = {2025}
}

The democratization of cobots makes them accessible for physically producing paintings and drawings in collaboration with artists. At the same time, large deep-learning models are becoming increasingly common tools for a variety of complex image generation tasks. We present a method that combines these two advancements by enabling gradient-based optimization of natural human-like motions guided by cost functions defined in image space. To this end, we use the sigma-lognormal model of human hand/arm movements with an adaptation that enables its use in conjunction with a differentiable vector graphics (DiffVG) renderer. We demonstrate how this pipeline can be used to generate feasible trajectories for a robot by combining image-driven objectives with a minimum-time smoothing criterion. We demonstrate applications with generation and robotic reproduction of synthetic graffiti as well as image abstraction.

@inproceedings{Berio25ROMAN,
  author = {Berio, D. and Clivaz, G. and Stroh, M. and Deussen, O. and Plamondon, R. and Calinon, S. and Leymarie, F. F.},
  booktitle = {Proc.{{IEEE}} Intl Symp.on Robot and Human Interactive Communication ({{Ro-Man}})},
  title = {Image-Driven Robot Drawing with Rapid Lognormal Movements},
  year = {2025}
}

Labor is a live human/robot painting installation that combines generative graphics techniques, robotic automation and traditional painting methods. It explores the role of embodied intelligence in artistic production. The resulting composition is a large-scale painting consisting of multiple individually painted tiles. The painting is based on an electron microscope image of a placenta, which is algorithmically processed into a series of parametric brushstrokes using a differentiable vector graphics pipeline. These strokes are then collaboratively painted using a 7-axis robotic arm equipped with custom paintbrushes. The project engages with the dual meaning of ''labor'': industrial production and childbirth, highlighting the often-overlooked importance of bodily knowledge in the arts but particularly in medical and technological contexts. The installation explores the balance between human intuition and algorithmic automation, emphasizing the importance of material constraints and the role of human artists in the creation a large-scale generative painting.

@inproceedings{GrayverLabor2025,
  author = {Grayver, Liat and Berio, Daniel and Herrmann, Inge and Notz, Adrian},
  booktitle = {{{ACM}}/{{EG}} Expressive Symposium - {{WICED}}: {{Eurographics}} Workshop on Intelligent Cinematography and Editing - Artworks, Posters, Demos},
  doi = {10.2312/exw.20251063},
  editor = {Berio, Daniel and Bruckert, Alexandre},
  isbn = {978-3-03868-272-1},
  publisher = {The Eurographics Association},
  title = {{{LABOR}}: {{Production}} of a Large-Scale Painting with a Robot},
  url = {https://diglib.eg.org/items/f5e96c76-c89e-40c2-ad27-654a845d6e03},
  year = {2025}
}

A word-as-image is a semantic typography technique where a word illustration presents a visualization of the meaning of the word, while also preserving its readability. We present a method to create word-as-image illustrations automatically. This task is highly challenging as it requires semantic understanding of the word and a creative idea of where and how to depict these semantics in a visually pleasing and legible manner. We rely on the remarkable ability of recent large pretrained language-vision models to distill textual concepts visually. We target simple, concise, black-and-white designs that convey the semantics clearly. We deliberately do not change the color or texture of the letters and do not use embellishments. Our method optimizes the outline of each letter to convey the desired concept, guided by a pretrained Stable Diffusion model. We incorporate additional loss terms to ensure the legibility of the text and the preservation of the style of the font. We show high quality and engaging results on numerous examples and compare to alternative techniques. Code and demo will be available at our project page.

@article{IluzWordAsImage2023,
  address = {New York, NY, USA},
  articleno = {151},
  author = {Iluz, Shir and Vinker, Yael and Hertz, Amir and Berio, Daniel and {Cohen-Or}, Daniel and Shamir, Ariel},
  doi = {10.1145/3592123},
  issn = {0730-0301},
  issue_date = {August 2023},
  journal = {ACM Transactions on Graphics},
  month = {July},
  number = {4},
  publisher = {Association for Computing Machinery},
  title = {Word-as-Image for Semantic Typography},
  url = {https://doi.org/10.1145/3592123},
  volume = {42},
  year = {2023}
}

We develop a method to automatically segment a font's glyphs into a set of overlapping and intersecting strokes with the aim of generating artistic stylizations. The segmentation method relies on a geometric analysis of the glyph's outline, its interior, and the surrounding areas and is grounded in perceptually informed principles and measures. Our method does not require training data or templates and applies to glyphs in a large variety of input languages, writing systems, and styles. It uses the medial axis, curvilinear shape features that specify convex and concave outline parts, links that connect concavities, and seven junction types. We show that the resulting decomposition in strokes can be used to create variations, stylizations, and animations in different artistic or design-oriented styles while remaining recognizably similar to the input font.

@article{berioStrokeStylesStrokebasedSegmentation2022,
  address = {New York, NY, USA},
  articleno = {28},
  author = {Berio, Daniel and Leymarie, Frederic Fol and Asente, Paul and Echevarria, Jose},
  doi = {10.1145/3505246},
  issn = {0730-0301},
  issue_date = {June 2022},
  journal = {ACM Transactions on Graphics},
  month = {April},
  number = {3},
  publisher = {Association for Computing Machinery},
  title = {{{StrokeStyles}}: {{Stroke-based}} Segmentation and Stylization of Fonts},
  url = {https://doi.org/10.1145/3505246},
  volume = {41},
  year = {2022}
}

A dominant theory of embodied aesthetic experience (Freedberg \& Gallese, 2007, Trends in Cognitive Sciences, 11, 197) posits that the appreciation of visual art is linked to the artist's movements when creating the artwork, yet a direct link between the kinematics of drawing actions and the aesthetics of drawing outcomes has not been experimentally demonstrated. Across four experiments, we measured aesthetic responses of students from arts and non-arts backgrounds to drawing movements generated from computational models of human writing. Experiment 1 demonstrated that human-like drawing movements with bell-shaped velocity profiles (Sigma Lognormal [SL] and Minimum Jerk [MJ]) are perceived as more natural and pleasant than movements with a uniform profile, and in both Experiments 1 and 2 movements that were perceived as more natural were also preferred. Experiment 3 showed that this effect persists if lower-level dynamic stimulus features are fully matched across experimental and control conditions. Furthermore, aesthetic preference for human-like movements were associated with greater perceptual fluency in Experiment 3, evidenced by unbiased estimations of the duration of natural movements. In Experiment 4, line drawings with visual features consistent with the dynamics of natural, human-like movements were preferred, but only by art students. Our findings directly link the aesthetics of human action to the visual aesthetics of drawings, but highlight the importance of incorporating artistic expertise into embodied accounts of aesthetic experience.

@article{Chamberlain2022-um,
  author = {Chamberlain, Rebecca and Berio, Daniel and Mayer, Veronika and Chana, Kirren and Leymarie, Frederic Fol and Orgs, Guido},
  journal = {British Journal of Psychology},
  langid = {english},
  month = {February},
  number = {1},
  pages = {105--130},
  publisher = {Wiley},
  title = {A Dot That Went for a Walk: {{People}} Prefer Lines Drawn with Human-like Kinematics},
  url = {https://bpspsychub.onlinelibrary.wiley.com/doi/10.1111/bjop.12527},
  volume = {113},
  year = {2022}
}

A dominant theory of embodied aesthetic experience (Freedberg \& Gallese, 2007, Trends in Cognitive Sciences, 11, 197) posits that the appreciation of visual art is linked to the artist's movements when creating the artwork, yet a direct link between the kinematics of drawing actions and the aesthetics of drawing outcomes has not been experimentally demonstrated. Across four experiments, we measured aesthetic responses of students from arts and non-arts backgrounds to drawing movements generated from computational models of human writing. Experiment 1 demonstrated that human-like drawing movements with bell-shaped velocity profiles (Sigma Lognormal [SL] and Minimum Jerk [MJ]) are perceived as more natural and pleasant than movements with a uniform profile, and in both Experiments 1 and 2 movements that were perceived as more natural were also preferred. Experiment 3 showed that this effect persists if lower-level dynamic stimulus features are fully matched across experimental and control conditions. Furthermore, aesthetic preference for human-like movements were associated with greater perceptual fluency in Experiment 3, evidenced by unbiased estimations of the duration of natural movements. In Experiment 4, line drawings with visual features consistent with the dynamics of natural, human-like movements were preferred, but only by art students. Our findings directly link the aesthetics of human action to the visual aesthetics of drawings, but highlight the importance of incorporating artistic expertise into embodied accounts of aesthetic experience.

@article{Chamberlain2021,
  author = {Chamberlain, Rebecca and Berio, Daniel and Mayer, Veronika and Chana, Kirren and Leymarie, Frederic Fol and Orgs, Guido},
  doi = {10.1111/bjop.12527},
  eprint = {https://bpspsychub.onlinelibrary.wiley.com/doi/pdf/10.1111/bjop.12527},
  journal = {British Journal of Psychology},
  number = {n/a},
  title = {A Dot That Went for a Walk: {{People}} Prefer Lines Drawn with Human-like Kinematics},
  url = {https://bpspsychub.onlinelibrary.wiley.com/doi/abs/10.1111/bjop.12527},
  volume = {n/a},
  year = {2021}
}

Our goal is to be able to reproduce computationally calligraphic traces, such as found in the art practices of graffiti and various forms of more traditional cal- ligraphy, while mimicking their production process. To this end, we propose a method that allows to reconstruct kinematics solely from the geometric sam- ples of handwritten traces in the form of parameters of the Sigma-Lognormal model. We ignore the kinematics possibly embedded in the data in order to treat online data and vector patterns with the same procedure. At the heart of our method, we develop a robust procedure to identify curvilinear shape features based on an analysis of local symmetry axes. These features determine the segmentation of a trace into circular arcs and guide an iterative reconstruction of the input kinematics and geometry in the form of Sigma-Lognormal parameters. We demonstrate how this parametrisation can be used to generate plausible kinematics for a static input trace, and how pa- rameter variations can be exploited to generate traces that resemble the ones seen in real instances of human made calligraphy and graffiti.

@incollection{Berio2020Lognormality,
  author = {Berio, Daniel and Leymarie, Frederic Fol and Plamondon, R{\~A}{\copyright}jean},
  booktitle = {The Lognormality Principle and Its Applications in E-{{Security}}, e-{{Learning}} and e-{{Health}}},
  chapter = {11},
  doi = {10.1142/9789811226830_0011},
  editor = {Plamondon, R{\~A}{\copyright}jean and Marcelli, Angelo and Ferrer, Miguel {\~A}ngel},
  eprint = {https://www.worldscientific.com/doi/pdf/10.1142/9789811226830\_0011},
  month = {December},
  pages = {237--268},
  publishers = {World Scientific},
  series = {Series in Machine Perception and Artificial Intelligence},
  title = {Kinematics Reconstruction of Static Calligraphic Traces from Curvilinear Shape Features},
  url = {https://doc.gold.ac.uk/autograff/post/papers/Berio-Lognormality2021.pdf},
  volume = {88},
  year = {2020}
}

We present a variant of the skeletal strokes algorithm aimed at mimicking the appearance of hand made graffiti art. It includes a unique fold-culling process that stylizes folds rather than eliminating them. We demonstrate how the stroke structure can be exploited to generate non-global layering and self-overlap effects like the ones that are typically seen in graffiti art and other related art forms like traditional calligraphy. The method produces vector output with no artificial artwork splits, patches or masks to render the non-global layering; each path of the vector output is part of the desired outline. The method lets users interactively generate a wide variety of stylised outputs.

@inproceedings{BerioExpressive2019,
  address = {Goslar, DEU},
  author = {Berio, Daniel and Asente, Paul and Echevarria, Jose and Fol Leymarie, Frederic},
  booktitle = {Proceedings of the 8th {{ACM}}/Eurographics Expressive Symposium on Computational Aesthetics and Sketch Based Interfaces and Modeling and Non-Photorealistic Animation and Rendering},
  doi = {10.2312/exp.20191076},
  pages = {51{\^a}{\texteuro}``59},
  publisher = {Eurographics Association},
  series = {Expressive '19},
  title = {Sketching and Layering Graffiti Primitives},
  url = {https://diglib.eg.org/server/api/core/bitstreams/65fb010c-96c5-479c-9f9a-a2cfed6850b0/content},
  year = {2019}
}

The chapter presents an approach for the interactive definition of curves and motion paths based on Gaussian mixture model (GMM) and optimal control. The input of our method is a mixture of multivariate Gaussians defined by the user, whose centers define a sparse sequence of keypoints, and whose covariances define the precision required to pass through these keypoints. The output is a dynamical system generating curves that are natural looking and reflect the kinematics of a movement, similar to that produced by human drawing or writing. In particular, the stochastic nature of the GMM combined with optimal control is exploited to generate paths with natural variations, which are defined by the user within a simple interactive interface. Several properties of the Gaussian mixture are exploited in this application. First, there is a direct link between multivariate Gaussian distributions and optimal control formulations based on quadratic objective functions (linear quadratic tracking), which is exploited to extend the GMM representation to a controller. We then exploit the option of tying the covariances in the GMM to modulate the style of the calligraphic trajectories. The approach is tested to generate curves and traces that are geometrically and dynamically similar to the ones that can be seen in art forms such as calligraphy or graffiti.

@article{BerioMixtures2020,
  author = {Berio, Daniel and Fol Leymarie, Frederic and Calinon, Sylvain},
  doi = {10.1007/978-3-030-23876-6_2},
  editor = {Bouguila, Nizar and Fan, Wentao},
  journal = {Mixture Models and Applications},
  pages = {23--38},
  publisher = {Springer},
  series = {Unsupervised and Semi-Supervised Learning},
  title = {Interactive Generation of Calligraphic Trajectories from Gaussian Mixtures},
  url = {https://doc.gold.ac.uk/autograff/post/papers/Berio-Mix2019.pdf},
  year = {2019}
}

We describe a practical application of the Sigma Lognormal model of handwriting movements for computer graphics appli- cations that require the interactive or procedural definition of artistic or calligraphic traces. The method allows to easily edit curves with physiologically plausible kinematics that can be exploited in order to generate expressive brush renderings, natural looking stroke animations and easily generate stylistic variations of a trace.

@inproceedings{BerioEG2018,
  address = {Goslar, DEU},
  author = {Berio, Daniel and Leymarie, Frederic Fol and Plamondon, R{\'e}jean},
  booktitle = {Proceedings of the 39th Annual European Association for Computer Graphics Conference: {{Short}} Papers},
  pages = {33--36},
  publisher = {Eurographics Association},
  series = {{{EG}}},
  title = {Expressive Curve Editing with the Sigma Lognormal Model},
  url = {https://research.gold.ac.uk/id/eprint/23409/1/eg-2018-short-lognormal.pdf},
  year = {2018}
}

We describe a methodology for the interactive definition of curves and motion paths using a stochastic formulation of optimal control. We demonstrate how the same optimization framework can be used in different ways to generate curves and traces that are geometrically and dynamically similar to the ones that can be seen in art forms such as calligraphy or graffiti art. The method provides a probabilistic description of trajectories that can be edited similarly to the control polygon typically used in the popular spline based methods. Furthermore, it also encapsulates movement kinematics, deformations and variability. The user is then provided with a simple interactive interface that can generate multiple movements and traces at once, by visually defining a distribution of trajectories rather than a single one. The input to our method is a sparse sequence of targets defined as multivariate Gaussians. The output is a dynamical system generating curves that are natural looking and reflect the kinematics of a movement, similar to that produced by human drawing or writing.

@inproceedings{BerioGI2017,
  address = {Edmonton, Canada},
  author = {Berio, D and Calinon, S. and Fol Leymarie, F.},
  booktitle = {Proceedings of Graphics Interface},
  month = {May},
  optseries = {GI 2017},
  publisher = {Canadian Human-Computer Communications Society},
  title = {Generating Calligraphic Trajectories with Model Predictive Control},
  url = {https://research.gold.ac.uk/20169/},
  year = {2017}
}

We present the Sigma Lognormal model as a potential tool for curve generation in computer graphics related applications. We discuss its extension and parameterisations for the interactive definition of handwriting, drawing and calligraphic trajectories. This results in an efficient trajectory synthesis method, that has a user interface similar to the ones commonly used with B\' ezier curves or splines, but with the added benefit of capturing the kinematics of human drawing or writing movements. Such kinematics produced by the model can then be exploited to generate realistic stroke animations or to facilitate expressive rendering methods.

@inproceedings{BerioIGS2017,
  author = {Berio, D. and Fol Leymarie, F. and R., Plamondon},
  booktitle = {18th Biennial Conference of the International Graphonomics Society},
  title = {Computer Aided Design of Handwriting Trajectories with the Kinematic Theory of Rapid Human Movements},
  url = {https://research.gold.ac.uk/id/eprint/20757/1/berio-igs2017.pdf},
  year = {2017}
}

We present a method for the interactive generation of stylised letters, curves and motion paths that are similar to the ones that can be observed in art forms such as graffiti and calligraphy. We define various stylisations of a letter form over a common geometrical structure, which is given by the spatial layout of a sparse sequence of targets. Different stylisations are then generated by optimising the trajectories of a dynamical system that tracks the target sequence. The evolution of the dynamical system is computed with a stochastic formulation of optimal control, in which each target is defined probabilistically as a multivariate Gaussian. The covariance of each Gaussian explicitly defines the variability as well as the curvilinear evolution of trajectory segments. Given this probabilistic formulation, the optimisation procedure results in a trajectory distribution rather than a single path. It is then possible to stochastically sample from the distribution an infinite number of dynamically and aesthetically consistent trajectories which mimic the variability that is typically observed in human drawing or writing. We further demonstrate how this system can be used together with a simple user interface in order to explore different stylisations of interactively or procedurally defined letters.

@inproceedings{BerioMPCMOCO2017,
  address = {London, UK},
  author = {Berio, D and Calinon, S. and Fol Leymarie, F.},
  booktitle = {{{ACM}} Proceedings of the 4th International Conference on Movement and Computing},
  month = {June},
  optaddress = {New York, NY, USA},
  optpublisher = {ACM},
  optseries = {MOCO '17},
  title = {Dynamic Graffiti Stylisation with Stochastic Optimal Control},
  url = {https://research.gold.ac.uk/20758/},
  year = {2017}
}

We propose a computational framework to learn stylisation patterns from example drawings or writings, and then generate new trajectories that possess similar stylistic qualities. We particularly focus on the generation and stylisation of trajectories that are similar to the ones that can be seen in calligraphy and graffiti art. Our system is able to extract and learn dynamic and visual qualities from a small number of user defined examples which can be recorded with a digitiser device, such as a tablet, mouse or motion capture sensors. Our system is then able to transform new user drawn traces to be kinematically and stylistically similar to the training examples. We implement the system using a Recurrent Mixture Density Network (RMDN) combined with a representation given by the parameters of the Sigma Lognormal model, a physiologically plausible model of movement that has been shown to closely reproduce the velocity and trace of human handwriting gestures.

@inproceedings{BerioRNNMOCO2017,
  address = {London, UK},
  author = {Berio, D and Akten, M and Fol Leymarie, F. and Grierson, M. and Plamondon, R.},
  booktitle = {Proc. of 4th Int'l Conf. on Movement Computing ({{MOCO}})},
  optbooktitle = {Proceedings of the 4th International Symposium on Movement and Computing},
  optpublisher = {ACM},
  optseries = {MOCO '17},
  title = {Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks},
  url = {https://research.gold.ac.uk/20759/},
  year = {2017}
}

We present an approach to generate rapid and fluid drawing movements on a compliant Baxter robot, by taking advantage of the kinematic redundancy and torque control capabilities of the robot. We concentrate on the task of reproducing graffiti-stylised letter-forms with a marker. For this purpose, we exploit a compact lognormal-stroke based representation of movement to generate natural drawing tra- jectories. An Expectation-Maximisation (EM) algorithm is used to iteratively improve tracking performance with low gain feedback control. The resulting system captures the aesthetic and dynamic features of the style under investigation and permits its reproduction with a compliant controller that is safe for users surrounding the robot.

@inproceedings{Berio16IROS,
  address = {Daejeon, Korea},
  author = {Berio, D. and Calinon, S. and Fol Leymarie, F.},
  booktitle = {Proc. {{IEEE}}/{{RSJ}} Intl Conf. on Intelligent Robots and Systems ({{IROS}})},
  month = {October},
  title = {Learning Dynamic Graffiti Strokes with a Compliant Robot},
  url = {https://publications.idiap.ch/downloads/papers/2016/Berio_IROS_2016.pdf},
  year = {2016}
}

In this paper we describe a system aimed at the generation and analysis of graffiti tags.We argue that the dynamics of the movement involved in generating tags is in large part - and at a higher degree with respect to many other visual art forms - determinant of their stylistic quality. To capture this notion computationally, we rely on a biophysically plausible model of handwriting gestures (the Sigma Lognormal Model proposed by R\'ejean Plamondon et al.) that permits the generation of curves which are aesthetically and kinetically similar to the ones made by a human hand when writing. We build upon this model and extend it in order to facilitate the interactive construction and manipulation of digital tags. We then describe a method that reconstructs any planar curve or a sequence of planar points with a set of corresponding model parameters. By doing so, we seek to recover plausible velocity and temporal information for a static trace. We present a number of applications of our system: (i) the interactive design of curves that closely resemble the ones typically observed in graffiti art; (ii) the stylisation and beautification of input point sequences via curves that evoke a smooth and rapidly executed movement; (iii) the generation of multiple instances of a synthetic tag from a single example. This last application is a step in the direction of our longer term plan of realising a system which is capable of automatically generating convincing images in the graffiti style space.

@inproceedings{Berio2015cae,
  author = {Berio, Daniel and Leymarie, Frederic Fol},
  booktitle = {Computational Aesthetics (Cae)},
  editor = {Rosin, Paul},
  month = {June},
  pages = {35--47},
  publisher = {Eurographics Association},
  title = {Computational Models for the Analysis and Synthesis of Graffiti Tag Strokes},
  url = {https://diglib.eg.org/items/d4fb0442-3202-4b82-99b1-06493c1e0366},
  year = {2015}
}