“说服自己相信错误的东西” 是不是去我执?相信“自认为正确”的东西很简单,但是反面却不是那么轻松。“见人信人话,见鬼信鬼话”,八面玲弄是为了取悦别人,但是“凡事都都信了“自己就轻松了。执着也就没了

[BLOG] 冰岛🇮🇸旅行攻略

Traveling plan for Iceland in Dec 2023. It is a pain to use Notion and Google Maps to plan the trip – Notion is too slow, compact and lumped together, and Google Maps is awkward to use for planning. Moreover, information is scattered everywhere, and it is really hard to find a good source of information and keep things organized.

Continue Reading »

[TECH] Make a Mini ASIC for Robots

A post of investigating the possibility of making my own ASIC chip. My initial thought was to design a custom Yubikey and use it as a shortcut key to awake AI agents; something like the iPhone 15 side button. Later I realized it might be more interesting to design a mini ASIC as a learning project, since I’ve never got a chance to get involved in the chip design process.

Continue Reading »

[BLOG] AGI 未来展望

2023 AI 元年差不多也接近尾声了。回望一下,从年初ChatGPT 的 hypes/vibes 不断刷屏,到各种多模态模型的惊艳表现,让大家有一种 “科技奇点” 已经到来,AGI即将实现的感觉;AI领域欣欣向荣的希望,伴随着就业市场低迷,通胀/经济衰退的悲观环境,让人有一种魔幻的感觉。

Continue Reading »

Bill Dally’s invited talk at Cornell. Bill is a senior big name in accelerator/CompArch since 1985. The talk primarily focuses on Nvidia’s efforts to enhance GPU performance, starting from Kepler in the early 2000s and extending to the recent Hopper H200, which includes improvements in low-precision and sparsity support. In addition to the technical aspects, an intriguing comment he made about AI caught my attention. It directly echoes Rich Sutton’s “Bitter Lessons” blog, emphasizing that AI model architectures remain simple, much like they were 50 years ago; the only significant change lies in the increased compute power at our disposal, making it possible for us to stack deeper layers. Modern hardware enables LLM-like Bayesian-based AI models to more effectively approximate the true distribution of the world.

[NOTE] Unix Command Book

Basic concepts in ML/AI algorithms, and some simple algorithms implementations. There are so many operators and variants of them; this post only covers the most basic ones.

Continue Reading »

Watched the final chapter of Attack on Titan (AOT) in double speed. Setting aside the emotional parts, there are a few things that left a deep impression on me. Firstly, the counterattack against nihilism: humans are just a material entity with the attribute of ‘procreation,’ and the so-called freedom is false. The cyclic nature of the game is repetitive and meaningless, yet the process of existence brings joy. However, this sense of small happiness, or existentialism, is merely shifting the center of coordinates from the universe to oneself. Another thing is conflict. Strife and combat exist among all living beings, so the Voluntary Human Extinction Movement (VHEMT) is a false proposition. Even if humans disappear, other life forms will continue to engage in conflict and destruction. Heroes may step forward to bring an end to war, but war itself will not cease.

[NOTE] Compilers and Programming Models

Classic compiler frameworks such as Clang, LLVM, MLIR, etc. AI optimizing compilers like Torch, TensorRT, TVM, XLA/HLO, JAX/Pallas, Triton, etc. Also includes wider topics such as programming models, and program synthesis techniques, such as High-Level Synthesis (HLS), target-specific domain specific languages, etc.

Continue Reading »

[NOTE] Nvidia GPU, CUDA and PTX

AI hardware accelerators, including FPGA, and other emerging ASICs like Google TPU, AMD AIE, AWS Trainium, etc. This note does not include details about the software stack, but only focuses on the hardware architecture and its design choices.

Continue Reading »

AI Algorithms and Operators

Basic concepts in ML/AI algorithms, and some simple algorithms implementations. There are so many operators and variants of them; this post only covers the most basic ones.

Continue Reading »