Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2151 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.2488. Min Act: -0.0000

Data Index: 1142401 (The Pile)

Max Activating Token Index: 609

Text #1

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.1057. Min Act: -0.0000

Data Index: 127095 (The Pile)

Max Activating Token Index: 96

Text #2

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.1898. Min Act: -0.0000

Data Index: 1041209 (The Pile)

Max Activating Token Index: 364

Text #3

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.1596. Min Act: -0.0000

Data Index: 552219 (The Pile)

Max Activating Token Index: 320

Text #4

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.1439. Min Act: -0.0000

Data Index: 232817 (The Pile)

Max Activating Token Index: 56

Text #5

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.1126. Min Act: -0.0000

Data Index: 1325364 (The Pile)

Max Activating Token Index: 696

Text #6

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.1028. Min Act: -0.0000

Data Index: 1375958 (The Pile)

Max Activating Token Index: 502

Text #7

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.0737. Min Act: -0.0000

Data Index: 1290897 (The Pile)

Max Activating Token Index: 129

Text #8

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.1687. Min Act: -0.0000

Data Index: 438385 (The Pile)

Max Activating Token Index: 330

Text #9

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.1030. Min Act: -0.0000

Data Index: 949984 (The Pile)

Max Activating Token Index: 1016

Text #10

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.0215. Min Act: -0.0000

Data Index: 128685 (The Pile)

Max Activating Token Index: 52

Text #11

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.1152. Min Act: -0.0000

Data Index: 531883 (The Pile)

Max Activating Token Index: 269

Text #12

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.1366. Min Act: -0.0000

Data Index: 1226745 (The Pile)

Max Activating Token Index: 833

Text #13

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.1206. Min Act: -0.0000

Data Index: 1127198 (The Pile)

Max Activating Token Index: 589

Text #14

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.1332. Min Act: -0.0000

Data Index: 1954959 (The Pile)

Max Activating Token Index: 70

Text #15

Max Range: 1.2488. Min Range: -1.2488

Max Act: 0.9695. Min Act: -0.0000

Data Index: 956899 (The Pile)

Max Activating Token Index: 711

Text #16

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.1160. Min Act: -0.0000

Data Index: 387294 (The Pile)

Max Activating Token Index: 690

Text #17

Max Range: 1.2488. Min Range: -1.2488

Max Act: 0.9523. Min Act: -0.0000

Data Index: 108349 (The Pile)

Max Activating Token Index: 155

Text #18

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.0684. Min Act: -0.0000

Data Index: 1396248 (The Pile)

Max Activating Token Index: 852

Text #19

Max Range: 1.2488. Min Range: -1.2488

Max Act: 1.0015. Min Act: -0.0000

Data Index: 201895 (The Pile)

Max Activating Token Index: 32