dst

dst (Decorated Syntax Tree) is a Go library that solves a well-known limitation of the standard go/ast package: comments are tracked by byte offset in the token.FileSet, so rearranging, replacing, or cloning nodes silently detaches them from the code they annotate. dst mirrors nearly every go/ast node type but attaches decorations — comments and blank-line spacing — directly to each node via a generated Decs struct with named attachment points, so annotations move with the node through any transformation.

The package exposes a Decorator/Restorer pair for round-tripping between ast and dst, optional import-block management via pluggable resolvers, a dstutil fork of astutil’s Apply for structured tree walking, and a Clone helper for safely reusing nodes elsewhere in a tree. It’s built for authors of code generators, refactoring tools, and linters with autofix that need to programmatically edit real-world Go source without destroying the developer’s existing comments and formatting.

What You Get

• A Decorator/Restorer pair for converting cleanly between go/ast and dst and back to formatted Go source
• Per-node Decs structs with named decoration attachment points (Start, End, Before, After, and node-specific points like Lparen/Rparen)
• Import-aware decoration via pluggable Resolver implementations (gotypes, guess, simple, and a go/packages-based resolver)
• The dstutil package, a fork of golang.org/x/tools/go/ast/astutil, providing Apply-based visitor tree rewriting for dst nodes
• A Clone helper that deep-copies any node while nulling Object/Scope pointers so clones never alias the original tree’s resolution data
• decorator.Load, a go/packages-based convenience function that loads and decorates every file in a package in one call

Common Use Cases

• Codemods and refactoring tools that rewrite Go source without losing the developer’s existing comments
• Code generators that patch or reorder declarations in hand-written files while preserving formatting
• Linters with autofix that need clean diffs instead of go/ast’s comment-detachment behavior
• Programmatic import-block management when generated code adds new qualified references

Under The Hood

Architecture dst represents Go source as a Decorated Syntax Tree that mirrors go/ast’s node types (see dst.go) but carries decoration info directly on each node instead of by file-position offset. Core files split cleanly by responsibility: dst.go and the generated decorations-types-generated.go / decorations-node-generated.go define the ~40 node types and their per-node Decs attachment points; decorator/decorator.go performs the ast-to-dst pass (NewDecorator, NewDecoratorWithImports for import-aware conversion via resolver.DecoratorResolver implementations under decorator/resolver); decorator/restorer.go performs the inverse dst-to-ast pass feeding go/format; clone-generated.go and clone.go implement deep-copy with explicit no-ops on Object/Scope pointers to prevent aliasing after cloning; dstutil/ forks golang.org/x/tools/go/ast/astutil to provide Apply-based tree walking compatible with dst nodes. Notably, gendst/ is an internal code generator (its own main.go, using github.com/dave/jennifer as a templating engine) that parses the real go/ast source to produce all the “-generated.go” files, keeping every node type in sync with upstream go/ast by codegen rather than hand maintenance — so the generator templates, not the generated files, are the actual source of truth for the node-type layer.

Tech Stack A pure Go module (go.mod targets Go 1.18) with a lean dependency set: golang.org/x/tools (for go/packages, and as the base astutil forks into dstutil), golang.org/x/mod as an indirect dependency, github.com/dave/jennifer for code generation templating inside gendst, github.com/sergi/go-diff used in tests to diff generated output against fixtures, and gopkg.in/src-d/go-billy.v4 providing an in-memory filesystem for Load-related test fixtures. There is no web or CLI framework in the library itself — gendst/main.go is a small internal build tool invoked via go:generate, not a user-facing binary. The build/test loop is go generate followed by go test ./...; CI is a legacy Travis config running only go test ./....

Code Quality The repository carries roughly two dozen _test.go files spread across the root, decorator, and dstutil packages (decorator_test.go, restorer_test.go, decorations_test.go, print_test.go, util_test.go, and more), covering round-trip parse/decorate/restore/print fidelity across many syntax shapes plus position preservation and resolver behavior. Error handling follows standard Go idiom (plain error returns), with a few intentional panics documented in Clone’s usage examples for programmer-error cases. No linter configuration (no golangci-lint file) is present in the repo, and CI runs tests only, without a vet or lint stage. Exported API surface (NewDecorator, Decorator, Restorer, Load, etc.) carries GoDoc comments consistent with standard-library conventions.

What Makes It Unique The core novel choice is attaching decorations directly to each syntax node through generated per-node Decs structs with named attachment points, rather than reconstructing comments from byte-offset comment maps the way go/ast does — this directly addresses a long-standing, specifically cited Go tooling limitation (golang/go issue #20744) where rearranging ast nodes silently detaches their comments. dst also folds import-block management into the same decorate/restore round trip via pluggable Resolver implementations, so qualified identifiers can be added to a tree without manually maintaining the import block by hand. This is a narrow, well-targeted solution rather than a generic framework, and it underlies a number of real-world Go code-generation and refactoring tools built on top of it.