Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4140 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.7774. Min Act: -0.0000

Data Index: 137099 (The Pile)

Max Activating Token Index: 177

Text #1

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.6984. Min Act: -0.0000

Data Index: 1391301 (The Pile)

Max Activating Token Index: 634

Text #2

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.5129. Min Act: -0.0000

Data Index: 160100 (The Pile)

Max Activating Token Index: 501

Text #3

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.5714. Min Act: -0.0000

Data Index: 1996393 (The Pile)

Max Activating Token Index: 124

Text #4

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.4414. Min Act: -0.0000

Data Index: 1165226 (The Pile)

Max Activating Token Index: 395

Text #5

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.5122. Min Act: -0.0000

Data Index: 1160588 (The Pile)

Max Activating Token Index: 526

Text #6

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.4268. Min Act: -0.0000

Data Index: 1833192 (The Pile)

Max Activating Token Index: 513

Text #7

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.5397. Min Act: -0.0000

Data Index: 1593437 (The Pile)

Max Activating Token Index: 571

Text #8

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.4357. Min Act: -0.0000

Data Index: 1593189 (The Pile)

Max Activating Token Index: 292

Text #9

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.4225. Min Act: -0.0000

Data Index: 1942537 (The Pile)

Max Activating Token Index: 64

Text #10

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.0970. Min Act: -0.0000

Data Index: 1419395 (The Pile)

Max Activating Token Index: 810

Text #11

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.3920. Min Act: -0.0000

Data Index: 58368 (The Pile)

Max Activating Token Index: 259

Text #12

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.3029. Min Act: -0.0000

Data Index: 1617031 (The Pile)

Max Activating Token Index: 194

Text #13

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.3631. Min Act: -0.0000

Data Index: 1761179 (The Pile)

Max Activating Token Index: 515

Text #14

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.3402. Min Act: -0.0000

Data Index: 423488 (The Pile)

Max Activating Token Index: 634

Text #15

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.3422. Min Act: -0.0000

Data Index: 474214 (The Pile)

Max Activating Token Index: 69

Text #16

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.3277. Min Act: -0.0000

Data Index: 810904 (The Pile)

Max Activating Token Index: 449

Text #17

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.4112. Min Act: -0.0000

Data Index: 1042975 (The Pile)

Max Activating Token Index: 586

Text #18

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.3338. Min Act: -0.0000

Data Index: 603347 (The Pile)

Max Activating Token Index: 827

Text #19

Max Range: 1.7774. Min Range: -1.7774

Max Act: 1.3200. Min Act: -0.0000

Data Index: 1348130 (The Pile)

Max Activating Token Index: 114