Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 692 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.6748. Min Act: -0.0000

Data Index: 1748666 (The Pile)

Max Activating Token Index: 845

Text #1

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.5781. Min Act: -0.0000

Data Index: 1849475 (The Pile)

Max Activating Token Index: 400

Text #2

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.6517. Min Act: -0.0000

Data Index: 964545 (The Pile)

Max Activating Token Index: 534

Text #3

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.6452. Min Act: -0.0000

Data Index: 1653693 (The Pile)

Max Activating Token Index: 523

Text #4

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.5506. Min Act: -0.0000

Data Index: 1720973 (The Pile)

Max Activating Token Index: 934

Text #5

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.6083. Min Act: -0.0000

Data Index: 1019260 (The Pile)

Max Activating Token Index: 634

Text #6

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.5085. Min Act: -0.0000

Data Index: 1446133 (The Pile)

Max Activating Token Index: 886

Text #7

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.5104. Min Act: -0.0000

Data Index: 1407636 (The Pile)

Max Activating Token Index: 496

Text #8

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.3776. Min Act: -0.0000

Data Index: 696797 (The Pile)

Max Activating Token Index: 231

Text #9

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.3487. Min Act: -0.0000

Data Index: 721743 (The Pile)

Max Activating Token Index: 167

Text #10

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.5581. Min Act: -0.0000

Data Index: 1863666 (The Pile)

Max Activating Token Index: 925

Text #11

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.2593. Min Act: -0.0000

Data Index: 873366 (The Pile)

Max Activating Token Index: 412

Text #12

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.3763. Min Act: -0.0000

Data Index: 1211178 (The Pile)

Max Activating Token Index: 522

Text #13

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.2486. Min Act: -0.0000

Data Index: 1036972 (The Pile)

Max Activating Token Index: 518

Text #14

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.2279. Min Act: -0.0000

Data Index: 788630 (The Pile)

Max Activating Token Index: 109

Text #15

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.2011. Min Act: -0.0000

Data Index: 957400 (The Pile)

Max Activating Token Index: 696

Text #16

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.1788. Min Act: -0.0000

Data Index: 805158 (The Pile)

Max Activating Token Index: 795

Text #17

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.1460. Min Act: -0.0000

Data Index: 1345383 (The Pile)

Max Activating Token Index: 522

Text #18

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.1393. Min Act: -0.0000

Data Index: 742852 (The Pile)

Max Activating Token Index: 140

Text #19

Max Range: 1.6748. Min Range: -1.6748

Max Act: 1.1972. Min Act: -0.0000

Data Index: 1307238 (The Pile)

Max Activating Token Index: 345