Typography Research Collection

[hackmd][github][github pages]

Typography is the cross between technology and liberal arts. This page is a research collection that includes computer graphics, computer vision, machine learning that related to typography. If you discover relevant new work, please feel free to contact the major maintainer, I-Sheng Fang. 🙂

Contents

• Large Language Model
• Vision Language Model
• Visual Text Generation and Editing
• Font Stye Transfer and Glyph Generation
  • NN Approach
  • Other Approach
• Struture Learning
• Dataset
• Other Application

Large Language Model

2025

• Font-Agent: Enhancing Font Understanding with Large Language Models
  • Yingxin Lai · Cuijie Xu · Haitian Shi · Guoqing Yang · Xiaoning Li · Zhiming Luo · Shaozi Li
  • CVPR 2025

Vision Language Model

• FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications
  • Yuki Tatsukawa, I-Chao Shen, Anran Qi, Yuki Koyama, Takeo Igarashi, Ariel Shamir
  • Eurographics 2024
  • [project page][paper][arxiv][code]

Visual Text Generation and Editing

2024

• MATIC: Multilingual Accurate Textual Image Customization via Joint Generative Artificial Intelligence
  • Chiao-Hsin Wu, I-Wei Lai
  • PacificVis 2025 Short Paper (Visnote) Track
• AnyText: Multilingual Visual Text Generation And Editing
  • Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He, Yifeng Geng∗, Xuansong Xie
  • ICLR 2024
  • [paper][code][demo]

2023

• GlyphControl: Glyph Conditional Control for Visual Text Generation
  • Yukang Yang, Dongnan Gui, Yuhui Yuan, Weicong Liang, Haisong Ding, Han Hu, Kai Chen
  • NeurIPS 2023
  • [paper][code]

• DiffUTE: Universal Text Editing Diffusion Model
  • Haoxing Chen, Zhuoer Xu1, Zhangxuan Gu*, Jun Lan, Xing Zheng, Yaohui Li, Changhua Meng, Huijia Zhu, Weiqiang Wang
    • ∗Corresponding author
  • NeurIPS 2023
  • [paper][code]

• TextDiffuser: Diffusion Models as Text Painters
  • Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, Furu Wei
    • *Equal Contribution
  • NeurIPS 2023
  • [project page][paper][code][demo][colab]

• GlyphDraw: Learning to Draw Chinese Characters in Image Synthesis Models Coherently
  • Jian Ma, Mingjun Zhao, Chen Chen, Ruichen Wang, Di Niu, Haonan Lu, Xiaodong Lin
  • arxiv 2023
  • [project page][paper][code]

• DeepFloyd IF
  • Inspired by “Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding”, NeurIPS2022
    • aka Imagen
  • [code]

• Character-Aware Models Improve Visual Text Rendering
  • arxiv
  • Rosanne Liu∗, Dan Garrette∗, Chitwan Saharia, William Chan, Adam Roberts, Sharan Narang, Irina Blok, RJ Mical, Mohammad Norouzi, Noah Constant∗
    • *Equal contribution
  • [paper]

• OCR-VQGAN: Taming Text-within-Image Generation
  • Juan A. Rodríguez, David Vázquez, Issam Laradji, Marco Pedersoli, Pau Rodríguez
  • WACV2023
  • [paper(CVF)][paper(arxiv)][code]

Font Stye Transfer and Glyph Generation

NN Approach

2024

• TypeDance: Creating Semantic Typographic Logos from Image through Personalized Generation
  • Shishi Xiao, Liangwei Wang, Xiaojuan Ma, Wei Zeng
  • CHI 2024
  • [arxiv] [code]

• Generate Like Experts: Multi-Stage Font Generation by Incorporating Font Transfer Process into Diffusion Models
  • Bin Fu, Fanghua Yu, Anran Liu, Zixuan Wang, Jie Wen, Junjun He, Yu Qiao
  • CVPR 2024
  • [CVF][code]

• VecFusion: Vector Font Generation with Diffusion
  • Vikas Thamizharasan, Difan Liu, Shantanu Agarwal, Matthew Fisher, Michael Gharbi, Oliver Wang, Alec Jacobson, Evangelos Kalogerakis
  • CVPR 2024
  • [arxiv] [project page]

• FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learning
  • Zhenhua Yang, Dezhi Peng, Yuxin Kong, Yuyi Zhang, Cong Yao, Lianwen Jin
  • AAAI 2024
  • [project page][arxiv][code][demo]

2023

• DS-Fusion: Artistic Typography via Discriminated and Stylized Diffusion
  • Maham Tanveer, Yizhi Wang, Ali Mahdavi-Amiri, Hao Zhang
  • ICCV2023
  • [project page][paper][code] [demo]

• Word-As-Image for Semantic Typography
  • Shir Iluz, Yael Vinker, Amir Hertz, Daniel Berio, Daniel Cohen-Or, Ariel Shamir
    • *Denotes equal contribution
  • SIGGRAPH 2023 - Honorable Mention Award
  • [project page][paper][code][demo]

• DeepVecFont-v2: Exploiting Transformers to Synthesize Vector Fonts with Higher Quality
  • Yuqing Wang, Yizhi Wang, Longhui Yu, Yuesheng Zhu, Zhouhui Lian
  • CVPR 2023
  • [paper(CVPR)][code]

2022

• StrokeStyles: Stroke-based Segmentation and Stylization of Fonts
  • Daniel Berio, Frederic Fol Leymarie, Paul Asente, Jose Echevarria
  • ACM Transactions on Graphics Volume 41 Issue 3 Article No.: 28 pp 1–21
  • [paper]

2021

• DeepVecFont: Synthesizing High-quality Vector Fonts via Dual-modality Learning
  • Yizhi Wang, Zhouhui Lian
  • SIGGRAPH Asia 2021
  • [project page][code][paper]

• DG-Font: Deformable Generative Networks for Unsupervised Font Generation
  • Yangchen Xie, Xinyuan Chen, Li Sun, Yue Lu
  • [paper(CVPR)][paper(arxiv)][code]
  • CVPR2021

• Multiple Heads are Better than One:Few-shot Font Generation with Multiple Localized Experts
  • Song Park, Sanghyuk Chun, Junbum Cha, Bado Lee, Hyunjung Shim
  • [paper][code]
  • ICCV 2021

• Font Style that Fits an Image – Font Generation Based on Image Context
  • [paper][code]
  • Taiga Miyazono, Brian Kenji Iwana, Daichi Haraguchi, and Seiichi Uchida
  • arxiv 2021

• Few-shot Font Generation with Localized Style Representations and Factorization
  • Song Park, Sanghyuk Chun, Junbum Cha, Bado Lee, Hyunjung Shim1
    • *: equal contribution
  • [project page] [paper][code]
  • AAAI 2021

• Handwritten Chinese Font Generation with Collaborative Stroke Refinement
  • [paper][WACV2021][code(dead)]
  • Chuan Wen, Jie Chang, Ya Zhang, Siheng Chen, Yanfeng Wang, Mei Han, Qi Tian
  • WACV 2021

2020

• RD-GAN: Few/Zero-Shot Chinese Character Style Transfer via Radical Decomposition and Rendering
  • Yaoxiong Huang, Mengchao He, Lianwen Jin, Yongpan Wang
  • [paper(ECVA)]
  • ECCV 2020

• Few-shot Compositional Font Generation with Dual Memory
  • Junbum Cha, Sanghyuk Chun, Gayoung Lee, Bado Lee, Seonghyeon Kim, Hwalsuk Lee.
  • [paper] [code]
  • ECCV 2020

• CalliGAN: Style and Structure-aware Chinese Calligraphy Character Generator
  • [paper][code]
  • Shan-Jean Wu, Chih-Yuan Yang and Jane Yung-jen Hsu
  • AI for Content Creation Workshop CVPR 2020.

2019

• A Learned Representation for Scalable Vector Graphics
  • [paper]
  • Raphael Gontijo Lopes, David Ha, Douglas Eck, Jonathon Shlens
  • ICCV 2019
  • ICLR workshop 2019

• Large-scale Tag-based Font Retrieval with Generative Feature Learning
  • [paper][page]
  • Tianlang Chen, Zhaowen Wang, Ning Xu, Hailin Jin, Jiebo Luo
  • ICCV 2019
• DynTypo: Example-based Dynamic Text Effects Transfer
  • [paper]
  • Yifang Men, Zhouhui Lian, Yingmin Tang, Jianguo Xiao
  • CVPR 2019
• Typography with Decor: Intelligent Text Style Transfer
  • Wenjing Wang, Jiaying Liu, Shuai Yang, and Zongming Guo
  • CVPR2019
  • [project page] [paper] [code]

  

• DeepGlyph
  • [websites]
• TET-GAN: Text Effects Transfer via Stylization and Destylization
  • [paper]
  • Shuai Yang, Jiaying Liu, Wenjing Wang, Zongming Guo
  • AAAI2019

• SCFont: Structure-Guided Chinese Font Generation via Deep Stacked Networks
  • [paper]
  • Yue Jiang, Zhouhui Lian*, Yingmin Tang, Jianguo Xiao
  • AAAI 2019

2018

• Coconditional Autoencoding Adversarial Networks for Chinese Font Feature Learning
  • [paper]
  • arxiv 2018
  • Zhizhan Zheng, Feiyun Zhang
• Separating Style and Content for Generalized Style Transfer
  • [paper][code]
  • Yexun Zhang, Ya Zhang, Wenbin Cai
  • CVPR 2018
  • Networks
  • results
• Deep Learning for Classical Japanese Literature
  • [paper][GitHub]
  • Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, David Ha
  • NeurIPS 2018
  • Kuzushiji-MNIST,Kuzushiji-49 and Kuzushiji-Kanji
  • Kuzushiji-MNIST
  • Kuzushiji-Kanji
• Multi-Content GAN for Few-Shot Font Style Transfer
  • [code][paper][blog]
  • Azadi, Samaneh, Matthew Fisher, Vladimir Kim, Zhaowen Wang, Eli Shechtman, and Trevor Darrell.
  • CVPR2018
• Learning to Write Stylized Chinese Characters by Reading a Handful of Examples
  • [paper]
  • Danyang Sun∗, Tongzheng Ren∗, Chongxuan Li, Hang Su†, Jun Zhu†
  • IJCAI 2018
  • SA-VAE

2017

• DCFont: An End-To-End Deep Chinese Font Generation System
  • [paper]
  • Juncheng Liu, Zhouhui Lian, Jianguo Xiao
  • SIGGRAPH Asia 2017

• zi2zi: Master Chinese Calligraphy with Conditional Adversarial Networks
  • [blog] [code(TensorFlow)] [code(pytorch)(xuan-li)] [code(pytorch)( EuphoriaYan)]
  • Yuchen Tian
  • 2017

  

2016

• Rewrite: Neural Style Transfer For Chinese Fonts
  • [code]
  • Yuchen Tian
  • 2016

  

• Automatic generation of large-scale handwriting fonts via style learning
  • [paper]
  • Zhouhui Lian, Bo Zhao, and Jianguo Xiao
  • SIGGRAPH ASIA 2016

• A Book from the Sky 天书: Exploring the Latent Space of Chinese Handwriting
  • [blog]
  • Gene Kogan

1993

• Letter Spirit: An Emergent Model of the Perception and Creation of Alphabetic Style
  • [paper]
  • Douglas Hofstadter, Gary McGraw
  • 1993

Other Approach

• Automatic Generation of Typographic Font from a Small Font Subset
  • [paper]
  • Tomo Miyazaki, Tatsunori Tsuchiya, Yoshihiro Sugaya, Shinichiro Omachi, Masakazu Iwamura, Seiichi Uchida, Koichi Kise
  • 2017

  

• Awesome Typography: Statistics-Based Text Effects Transfer
  • [paper][code]
  • Shuai Yang, Jiaying Liu, Zhouhui Lian and Zongming Guo
  • CVPR 2017

  

• FlexyFont: Learning Transferring Rules for Flexible Typeface Synthesis
  • [paper]
  • H. Q. Phan, H. Fu, and A. B. Chan
  • 2015 Computer Graphics Forum

  

• Automatic shape morphing for Chinese characters
  • [paper]
  • Zhouhui Lian, and Zhouhui Lian
  • SIGGRAPH Asia 2012
  • 
• Easy generation of personal Chinese handwritten fonts
  • [paper]
  • Baoyao Zhou, Weihong Wang, and Zhanghui Chen
  • 2011 ICME(IEEE International Conference on Multimedia and Expo)

  

Struture Learning

• Deep Factorization of Style and Structure in Fonts
  • [paper]
  • Nikita Srivatsan, Jonathan T. Barron, Dan Klein, Taylor Berg-Kirkpatrick
  • EMNLP 2019
• Analyzing 50k fonts using deep neural networks
  • [blog]
  • Erik Bernhardsson
  • 2016

  

• Learning a Manifold of Fonts
  • [paper][project(demo)]
  • Neill D.F. Campbell, and Jan Kautz
  • SIGGRAPH 2014

  

Dataset

• Chinese Handwriting Recognition Competition 2013
  • [paper][download]

• Deep Learning for Classical Japanese Literature
  • [paper][GitHub]
  • Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, David Ha
  • NeurIPS 2018
  • Kuzushiji-MNIST,Kuzushiji-49 and Kuzushiji-Kanji
  • Kuzushiji-MNIST
  • Kuzushiji-Kanji

Other Application

• FontCode: Embedding Information in Text Documents Using Glyph Perturbation
  • [paper][project page]
  • Chang Xiao, Cheng Zhang, Changxi Zheng
  • SIGGRAPH 2018