kid

package module
v1.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 25, 2025 License: MIT Imports: 8 Imported by: 0

README

GitHub go.mod Go versiongodocTestCoverage

kid

Package kid (K-sortable ID) provides a goroutine-safe generator of short (10 byte binary, 16 bytes when base32 encoded), url-safe, k-sortable unique IDs.

The 10-byte binary representation of an ID is composed of:

  • 6-byte value representing Unix time in milliseconds
  • 2-byte sequence, and,
  • 2-byte random value.

IDs encode (base32) as 16-byte url-friendly strings that look like:

06bqj05bhh2lcbdb

kid.ID features

  • Size: 10 bytes as binary, 16 bytes if stored/transported as an encoded string.
  • Timestamp + sequence is guaranteed to be unique for each call to New().
  • 2 bytes of trailing randomness to avoid counter-based attacks.
  • K-orderable in both binary and base32 encoded representations.
  • URL-friendly custom encoding without the vowels a, i, o, and u.
  • Automatic (un)/marshalling for SQL and JSON.
  • cmd/kid tool for ID generation and introspection.

Example usage

func main() {
    id := kid.New()
	  fmt.Printf("%s %s %03v\n", id, id.String(), id[:])
	  // Example output: 06bq7xhnr03mlz6r 06bq7xhnr03mlz6r [001 149 115 246 021 192 007 073 252 216]

	  id, err := kid.FromString("06bq7xhnr03mlz6r")
	  if err != nil {
	  	// do something
	  }
	  fmt.Printf("%s %s %03v\n", id, id.String(), id[:])
	  // Output: 06bq7xhnr03mlz6r 06bq7xhnr03mlz6r [001 149 115 246 021 192 007 073 252 216]
}

Acknowledgments

  • While the ID payload differs greatly, the API and much of this package borrows heavily from github.com/rs/xid, a zero-configuration globally-unique ID generator.

  • Unique timestamp+sequence pairs are generated by the github.com/google/uuid getV7Time() algorithm.

Uniqueness

Each call to kid.New() is guaranteed to return a unique ID with a timestamp+sequence greater than any previous call.

To satisfy whether kid.IDs are unique, run eval/uniqcheck/main.go:

$ go run eval/uniqcheck/main.go -count 2000000 -goroutines 20
# example  output:
Generating 2,000,000 IDs per 20 goroutines:
Total keys: 40,000,000. Keys in last time tick: 1,380. Number of dupes: 0

Or, at the command line, produce IDs and use OS utilities to check (single-threaded):

$ go install github.com/mwyvr/kid/cmd/kid@latest
$ kid -c 2000000 | sort | uniq -d
// None output

CLI

Package kid also provides a tool for id generation and inspection:

$ kid
06bpwm8x107evvh9

$ kid -c 2
06bpwm3hkm371gz4
06bpwm3hkm3d5ezr

# produce 4 and inspect
kid $(kid -c 4)
06bpwlvhb86bypp7 ts:1741312454738 seq:3247 rnd:23239 2025-03-07 01:54:14.738 +0000 UTC ID{  0x1, 0x95, 0x6e, 0x4f, 0x70, 0x52,  0xc, 0xaf, 0x5a, 0xc7 }
06bpwlvhb86gcdw6 ts:1741312454738 seq:3317 rnd:45958 2025-03-07 01:54:14.738 +0000 UTC ID{  0x1, 0x95, 0x6e, 0x4f, 0x70, 0x52,  0xc, 0xf5, 0xb3, 0x86 }
06bpwlvhb86gkmks ts:1741312454738 seq:3320 rnd:53817 2025-03-07 01:54:14.738 +0000 UTC ID{  0x1, 0x95, 0x6e, 0x4f, 0x70, 0x52,  0xc, 0xf8, 0xd2, 0x39 }
06bpwlvhb86gmb73 ts:1741312454738 seq:3322 rnd:10467 2025-03-07 01:54:14.738 +0000 UTC ID{  0x1, 0x95, 0x6e, 0x4f, 0x70, 0x52,  0xc, 0xfa, 0x28, 0xe3 }

Change Log

  • 2025-03-24 Drop minimum supported Go version to 1.23, thanks to heads up from @sergeevabc.
  • 2025-03-08 v1.2.0 released.
  • 2025-03-06 Forked rid in favour of kid for true k-sortability, requiring a new ID payload, now expected to remain static. Improved code coverage and documentation.

Contributing

Contributions are welcome.

Package Comparisons

kid was born out of a desire for a short, k-sortable unique ID where global uniqueness or inter-process ID generation coordination is not required.

A comparison of various Go ID generators:

Package BLen ELen K-Sort Encoded ID and Next Unique Components
mwyvr/kid 10 16 true 06bwz2qyzm14d070
06bwz2qyzm14fnte
06bwz2qyzm14hxmf
06bwz2qyzm14kdl1
unique (ts(ms) + sequence) + crypto/rand 6 byte ts(millisecond) : 2 byte sequence : 2 byte random
rs/xid 12 20 true cvhjc0tq9fa75iaa3d00
cvhjc0tq9fa75iaa3d0g
cvhjc0tq9fa75iaa3d10
cvhjc0tq9fa75iaa3d1g
ts(sec) + machineID + pid + counter 4 byte ts(sec) : 2 byte mach ID : 2 byte pid : 3 byte monotonic counter
segmentio/ksuid 20 27 true 2upRtyliBRn6UnfS2RsdkEIhqbg
2upRu1TTpojt5KQDykjTjreGXGE
2upRu0IZ0RbjFMSS1lb0Io3aQ8A
2upRu2AjlZoy3rnU6MJdqSuDs1H
ts + crypto/rand 4 byte ts(sec) : 16 byte random
google/uuid V4 16 36 false f03fed10-c632-4d06-95b5-6783796e6aaa
1c6c044b-66db-44e8-ac45-0e4e358dcc1f
9b26d7cd-d85d-4696-beec-0483d6446e7f
f2cf8c26-4e3c-4612-865c-2270883456fd
crypt/rand v4: 122 bits random; 6 bits embedding version & variant
google/uuid V7 16 36 true 0195cf8a-fefd-725a-8655-0910a814aa54
0195cf8a-fefd-725b-b7ee-b178436e1aa0
0195cf8a-fefd-725e-b663-bc0ae6ee45f9
0195cf8a-fefd-725f-8936-57d2937b3ecc
ts(ms) + crypt/rand v7: 16 bytes : 48 bits time, 12 bits sequence, 6 bits version/variant, 62 bits random
chilts/sid 16 23 true 1WlBXfeKc31-1L10__76tcP
1WlBXfeKcCQ-4_us_ZM7ZtY
1WlBXfeKcMr-0HIZhTn1J_0
1WlBXfeKcUN-4AYyBPKLA5P
ts + math/rand 8 byte ts(nanosecond) 8 byte random
matoous/go-nanoid/v2 21 21 false iaGKMTIcslXAPNkbIp4ho
hM2E2H2y56NtaljmWCfNs
SpTDFWVxLW_Rrk_S93A87
kPxkTozTenuSfwAO0s9rb
ts + crypto/rand 21 byte rand (adjustable)
sony/sonyflake 16 29 true GU2TSMZXGYYTCNZVGYYDANBZHEZTA
GU2TSMZXGYYTCNZVGYYDCMJVGQ3DM
GU2TSMZXGYYTCNZVGYYDCOBRGAYDE
GU2TSMZXGYYTCNZVGYYDENBWGUZTQ
ts + counter 39 bit ts(10msec) 8 bit seq, 16 bit mach id
oklog/ulid 16 26 true 01JQ7RNZQXCZ605958V4E9XMF0
01JQ7RNZQXK35FDQBEQM86VMEF
01JQ7RNZQXJV8JBXKFE1VVPJJT
01JQ7RNZQX1GJ4EQSG1KC3RY8N
ts + user-definable rand src 6 byte ts(ms) : 10 byte monotonic counter random init per ts(ms)
kjk/betterguid 17 20 true -OMEXjvxqWZMbCF4xNP6
-OMEXjvxqWZMbCF4xNP7
-OMEXjvxqWZMbCF4xNP8
-OMEXjvxqWZMbCF4xNP9
ts + rand-init counter 8 byte ts(ms) : 9 byte counter random init per ts(ms)

Package Benchmarks

A benchmark suite comparing some of the above-noted packages can be found in eval/bench/bench_test.go. All runs were done with scaling_governor set to performance:

echo "performance" | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
$ go test -cpu 1,2,4,8,16,32 -test.benchmem -bench .
goos: linux
goarch: amd64
pkg: github.com/mwyvr/kid/eval/bench
cpu: Intel(R) Core(TM) i9-14900K
BenchmarkKid                	25668646	       44.42 ns/op	      0 B/op	      0 allocs/op
BenchmarkKid-2              	24071296	       50.45 ns/op	      0 B/op	      0 allocs/op
BenchmarkKid-4              	15894621	       75.01 ns/op	      0 B/op	      0 allocs/op
BenchmarkKid-8              	12931524	       95.28 ns/op	      0 B/op	      0 allocs/op
BenchmarkKid-16             	10273124	      114.5 ns/op	      0 B/op	      0 allocs/op
BenchmarkKid-32             	8641492	      138.6 ns/op	      0 B/op	      0 allocs/op
BenchmarkXid                	39413270	       28.53 ns/op	      0 B/op	      0 allocs/op
BenchmarkXid-2              	42305062	       27.39 ns/op	      0 B/op	      0 allocs/op
BenchmarkXid-4              	42286348	       27.95 ns/op	      0 B/op	      0 allocs/op
BenchmarkXid-8              	40082140	       29.77 ns/op	      0 B/op	      0 allocs/op
BenchmarkXid-16             	36853810	       32.59 ns/op	      0 B/op	      0 allocs/op
BenchmarkXid-32             	58261640	       21.51 ns/op	      0 B/op	      0 allocs/op
BenchmarkKsuid              	15683168	       75.23 ns/op	      0 B/op	       0 allocs/op
BenchmarkKsuid-2            	14172432	       84.61 ns/op	      0 B/op	       0 allocs/op
BenchmarkKsuid-4            	11290425	      101.4 ns/op	      0 B/op	       0 allocs/op
BenchmarkKsuid-8            	9389043	      119.3 ns/op	      0 B/op	       0 allocs/op
BenchmarkKsuid-16           	7758454	      153.2 ns/op	      0 B/op	       0 allocs/op
BenchmarkKsuid-32           	6843870	      181.9 ns/op	      0 B/op	       0 allocs/op
BenchmarkGoogleUuid         	24592844	       47.72 ns/op	     16 B/op	       1 allocs/op
BenchmarkGoogleUuid-2       	29907156	       37.78 ns/op	     16 B/op	       1 allocs/op
BenchmarkGoogleUuid-4       	36956997	       31.78 ns/op	     16 B/op	       1 allocs/op
BenchmarkGoogleUuid-8       	44705392	       29.42 ns/op	     16 B/op	       1 allocs/op
BenchmarkGoogleUuid-16      	38979994	       33.91 ns/op	     16 B/op	       1 allocs/op
BenchmarkGoogleUuid-32      	43177587	       28.15 ns/op	     16 B/op	       1 allocs/op
BenchmarkGoogleUuidV7       	14226376	       83.90 ns/op	     16 B/op	       1 allocs/op
BenchmarkGoogleUuidV7-2     	12127534	       89.25 ns/op	     16 B/op	       1 allocs/op
BenchmarkGoogleUuidV7-4     	12975457	       92.48 ns/op	     16 B/op	       1 allocs/op
BenchmarkGoogleUuidV7-8     	12842493	       95.62 ns/op	     16 B/op	       1 allocs/op
BenchmarkGoogleUuidV7-16    	10053843	      119.1 ns/op	     16 B/op	       1 allocs/op
BenchmarkGoogleUuidV7-32    	7877816	      152.3 ns/op	     16 B/op	       1 allocs/op
BenchmarkUlid               	19653237	       60.15 ns/op	     16 B/op	       1 allocs/op
BenchmarkUlid-2             	29194627	       39.72 ns/op	     16 B/op	       1 allocs/op
BenchmarkUlid-4             	42736906	       29.85 ns/op	     16 B/op	       1 allocs/op
BenchmarkUlid-8             	38114436	       30.23 ns/op	     16 B/op	       1 allocs/op
BenchmarkUlid-16            	39977670	       34.00 ns/op	     16 B/op	       1 allocs/op
BenchmarkUlid-32            	41136363	       28.09 ns/op	     16 B/op	       1 allocs/op
BenchmarkBetterguid         	25478740	       46.51 ns/op	     24 B/op	       1 allocs/op
BenchmarkBetterguid-2       	24268438	       47.96 ns/op	     24 B/op	       1 allocs/op
BenchmarkBetterguid-4       	18773090	       64.66 ns/op	     24 B/op	       1 allocs/op
BenchmarkBetterguid-8       	14096698	       81.05 ns/op	     24 B/op	       1 allocs/op
BenchmarkBetterguid-16      	10897140	      110.5 ns/op	     24 B/op	       1 allocs/op
BenchmarkBetterguid-32      	9171368	      132.2 ns/op	     24 B/op	       1 allocs/op
PASS
ok  	github.com/mwyvr/kid/eval/bench	53.240s

Documentation

Overview

Package kid (K-sortable ID) provides a goroutine-safe generator of short (10 byte binary, 16 bytes when base32 encoded), url-safe, k-sortable unique IDs.

The 10-byte binary representation of an ID is composed of:

  • 6-byte value representing Unix time in milliseconds
  • 2-byte sequence, and,
  • 2-byte random value.

IDs encode (base32) as 16-byte url-friendly strings.

kid.ID features:

  • Size: 10 bytes as binary, 16 bytes if stored/transported as an encoded string.
  • Timestamp + sequence is guaranteed to be unique.
  • 2 bytes of trailing randomness
  • K-orderable in both binary and base32 encoded representations.
  • URL-friendly custom encoding without the vowels a, i, o, and u.
  • Automatic (un)/marshalling for SQL and JSON.
  • The cmd/kid tool for ID generation and introspection.

Example usage:

func main() {
    id := kid.New()
	  fmt.Printf("%s %s %03v\n", id, id.String(), id[:])
	  // Example output: 06bq7xhnr03mlz6r 06bq7xhnr03mlz6r [001 149 115 246 021 192 007 073 252 216]

	  id, err := kid.FromString("06bq7xhnr03mlz6r")
	  if err != nil {
	  	// do something
	  }
	  fmt.Printf("%s %s %03v\n", id, id.String(), id[:])
	  // Output: 06bq7xhnr03mlz6r 06bq7xhnr03mlz6r [001 149 115 246 021 192 007 073 252 216]
}

Acknowledgments:

While the ID payload differs greatly, the API and much of this package borrows heavily from https://github.com/rs/xid, a zero-configuration globally-unique ID generator. ID unique timestamp+sequence pairs are generated from the google/uuidV7 getV7Time() algorithm.

Index

Examples

Constants

This section is empty.

Variables

View Source
var (

	// ErrInvalidID represents an error state, typically when decoding invalid input
	ErrInvalidID = errors.New("kid: invalid id")
)

Functions

func Sort

func Sort(ids []ID)

Sort sorts an array of IDs in place.

Types

type ID

type ID [rawLen]byte

ID represents a unique identifier

func FromBytes

func FromBytes(b []byte) (ID, error)

FromBytes copies []bytes into an ID value. Only a length-check is performed.

func FromString

func FromString(str string) (ID, error)

FromString decodes a base32-encoded string to return an ID.

Example
id, err := FromString("03f6nlxczw0018fz")
if err != nil {
	panic(err)
}
fmt.Println(id.Timestamp(), id.Random())
Output:

946684799999 41439

func New

func New() (id ID)

New generates a new unique ID.

This function is goroutine-safe. IDs are composed of:

  • 6 bytes, timestamp, a Unix time in milliseconds
  • 2 bytes, sequence, a derived value ensuring uniqueness and order
  • 2 bytes, random value provided by crypto/rand

K-orderable: Each subsequent call to New() is guaranteed to produce an ID having a timestamp + sequence value greater than the previously generated ID.

Example

examples

id := New()
fmt.Printf(`ID:
    String()  %s
    Timestamp() %d
    Sequence() %d
    Random()  %d 
    Time()    %v
    Bytes()   %3v\n`, id.String(), id.Timestamp(), id.Sequence(), id.Random(), id.Time().UTC(), id.Bytes())

func (ID) Bytes

func (id ID) Bytes() []byte

Bytes returns the binary representation of id, which is simply id[:].

func (ID) Compare

func (id ID) Compare(other ID) int

Compare makes IDs k-sortable, behaving like `bytes.Compare`, returning 0 if two IDs are identical, -1 if the current ID is less than the other, and 1 if current ID is greater than other.

Note: only the first 8 bytes of the two IDs (timestamp+sequence) are compared.

func (ID) Encode

func (id ID) Encode(dst []byte) []byte

Encode the id using base32 encoding, writing 16 bytes to dst and return it.

func (ID) IsNil

func (id ID) IsNil() bool

IsNil returns true if ID == nilID.

func (ID) IsZero

func (id ID) IsZero() bool

IsZero is an alias of is IsNil.

func (ID) MarshalJSON

func (id ID) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface.

A json value will always be returned; as a nilID or any other binary ID will always encode, error will always be nil.

https://golang.org/pkg/encoding/json/#Marshaler

func (ID) MarshalText

func (id ID) MarshalText() ([]byte, error)

MarshalText implements `encoding.TextMarshaler`.

As any ID value will always encode, error is always nil. https://golang.org/pkg/encoding/#TextMarshaler

func (ID) Random

func (id ID) Random() int32

Random returns the two-byte random component of the ID.

func (*ID) Scan

func (id *ID) Scan(value any) error

Scan implements the sql.Scanner interface. https://pkg.go.dev/database/sql#Scanner

func (ID) Sequence

func (id ID) Sequence() int32

Sequence returns the ID sequence.

func (ID) String

func (id ID) String() string

String implements `fmt.Stringer`, returning id as a base32 encoded string using the kid custom character set. https://pkg.go.dev/fmt#Stringer

func (ID) Time

func (id ID) Time() time.Time

Time returns the ID's timestamp as a Time value with millisecond resolution and location set to UTC

func (ID) Timestamp

func (id ID) Timestamp() int64

Timestamp returns the timestamp component of id as milliseconds since the Unix epoch. Go timestamps are at location UTC.

func (*ID) UnmarshalJSON

func (id *ID) UnmarshalJSON(b []byte) error

UnmarshalJSON implements the json.Unmarshaler interface. https://golang.org/pkg/encoding/json/#Unmarshaler

func (*ID) UnmarshalText

func (id *ID) UnmarshalText(text []byte) error

UnmarshalText implements `encoding.TextUnmarshaler`, and performs a sanity check on text.

Note: decode() is only called from here and should never fail. https://pkg.go.dev/encoding#TextUnmarshaler

func (ID) Value

func (id ID) Value() (driver.Value, error)

Value implements package sql's driver.Valuer. https://pkg.go.dev/database/sql/driver#Valuer

Directories

Path Synopsis
cmd
kid command
A utility to generate or inspect kid.IDs.
A utility to generate or inspect kid.IDs.
eval
compare command
Package main produces for comparison purposes a markdown formatted table illustrating key differences between a number of unique ID packages.
Package main produces for comparison purposes a markdown formatted table illustrating key differences between a number of unique ID packages.
uniqcheck command
Package main provides a test to determine if ID generation delivers unique IDs for Go applications utilizing concurrency.
Package main provides a test to determine if ID generation delivers unique IDs for Go applications utilizing concurrency.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL