Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 465 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.9184. Min Act: -0.0000

Data Index: 1139347 (The Pile)

Max Activating Token Index: 300

Text #1

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.6851. Min Act: -0.0000

Data Index: 1645108 (The Pile)

Max Activating Token Index: 162

Text #2

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.6269. Min Act: -0.0000

Data Index: 1512928 (The Pile)

Max Activating Token Index: 327

Text #3

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.5864. Min Act: -0.0000

Data Index: 1174254 (The Pile)

Max Activating Token Index: 252

Text #4

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.5627. Min Act: -0.0000

Data Index: 109071 (The Pile)

Max Activating Token Index: 845

Text #5

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.5524. Min Act: -0.0000

Data Index: 1725296 (The Pile)

Max Activating Token Index: 700

Text #6

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.5512. Min Act: -0.0000

Data Index: 1045086 (The Pile)

Max Activating Token Index: 583

Text #7

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.5208. Min Act: -0.0000

Data Index: 1279548 (The Pile)

Max Activating Token Index: 956

Text #8

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.5448. Min Act: -0.0000

Data Index: 1712453 (The Pile)

Max Activating Token Index: 109

Text #9

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.5306. Min Act: -0.0000

Data Index: 1438892 (The Pile)

Max Activating Token Index: 814

Text #10

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.5337. Min Act: -0.0000

Data Index: 1895089 (The Pile)

Max Activating Token Index: 657

Text #11

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.4914. Min Act: -0.0000

Data Index: 1599948 (The Pile)

Max Activating Token Index: 328

Text #12

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.5016. Min Act: -0.0000

Data Index: 502348 (The Pile)

Max Activating Token Index: 516

Text #13

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.5193. Min Act: 0.0000

Data Index: 1480145 (The Pile)

Max Activating Token Index: 200

Text #14

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.5080. Min Act: -0.0000

Data Index: 279271 (The Pile)

Max Activating Token Index: 32

Text #15

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.4899. Min Act: 0.0000

Data Index: 1089305 (The Pile)

Max Activating Token Index: 606

Text #16

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.5015. Min Act: -0.0000

Data Index: 1313219 (The Pile)

Max Activating Token Index: 787

Text #17

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.4948. Min Act: -0.0000

Data Index: 724364 (The Pile)

Max Activating Token Index: 410

Text #18

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.5179. Min Act: 0.0001

Data Index: 227279 (The Pile)

Max Activating Token Index: 976

Text #19

Max Range: 0.9184. Min Range: -0.9184

Max Act: 0.4791. Min Act: -0.0000

Data Index: 1934148 (The Pile)

Max Activating Token Index: 981