Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3517 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.6824. Min Act: -0.0000

Data Index: 1469403 (The Pile)

Max Activating Token Index: 697

Text #1

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.5683. Min Act: -0.0000

Data Index: 751478 (The Pile)

Max Activating Token Index: 255

Text #2

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.4704. Min Act: -0.0000

Data Index: 1718465 (The Pile)

Max Activating Token Index: 459

Text #3

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.4500. Min Act: -0.0000

Data Index: 1254838 (The Pile)

Max Activating Token Index: 48

Text #4

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.3866. Min Act: -0.0000

Data Index: 432184 (The Pile)

Max Activating Token Index: 850

Text #5

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.2670. Min Act: -0.0000

Data Index: 1458550 (The Pile)

Max Activating Token Index: 50

Text #6

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.3979. Min Act: -0.0000

Data Index: 1320972 (The Pile)

Max Activating Token Index: 538

Text #7

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.3074. Min Act: -0.0000

Data Index: 329473 (The Pile)

Max Activating Token Index: 214

Text #8

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.3469. Min Act: -0.0000

Data Index: 1765813 (The Pile)

Max Activating Token Index: 29

Text #9

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.3349. Min Act: -0.0000

Data Index: 943445 (The Pile)

Max Activating Token Index: 509

Text #10

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.3010. Min Act: -0.0000

Data Index: 518365 (The Pile)

Max Activating Token Index: 160

Text #11

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.3224. Min Act: -0.0000

Data Index: 219040 (The Pile)

Max Activating Token Index: 251

Text #12

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.3117. Min Act: -0.0000

Data Index: 1679173 (The Pile)

Max Activating Token Index: 769

Text #13

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.3590. Min Act: -0.0000

Data Index: 462184 (The Pile)

Max Activating Token Index: 156

Text #14

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.2621. Min Act: -0.0000

Data Index: 1169057 (The Pile)

Max Activating Token Index: 679

Text #15

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.3713. Min Act: -0.0000

Data Index: 1588844 (The Pile)

Max Activating Token Index: 1012

Text #16

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.4505. Min Act: -0.0000

Data Index: 1149386 (The Pile)

Max Activating Token Index: 508

Text #17

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.2787. Min Act: -0.0000

Data Index: 506078 (The Pile)

Max Activating Token Index: 105

Text #18

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.2601. Min Act: -0.0000

Data Index: 1313811 (The Pile)

Max Activating Token Index: 892

Text #19

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.1983. Min Act: -0.0000

Data Index: 31453 (The Pile)

Max Activating Token Index: 993