Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2570 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.8980. Min Act: -0.0000

Data Index: 1566317 (The Pile)

Max Activating Token Index: 293

Text #1

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.9123. Min Act: -0.0000

Data Index: 606450 (The Pile)

Max Activating Token Index: 735

Text #2

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.7931. Min Act: -0.0000

Data Index: 25311 (The Pile)

Max Activating Token Index: 156

Text #3

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.7369. Min Act: -0.0000

Data Index: 1692429 (The Pile)

Max Activating Token Index: 563

Text #4

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.7181. Min Act: -0.0000

Data Index: 1307231 (The Pile)

Max Activating Token Index: 367

Text #5

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.6902. Min Act: -0.0000

Data Index: 408236 (The Pile)

Max Activating Token Index: 274

Text #6

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.7306. Min Act: -0.0000

Data Index: 1279893 (The Pile)

Max Activating Token Index: 598

Text #7

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.7153. Min Act: -0.0000

Data Index: 1827229 (The Pile)

Max Activating Token Index: 255

Text #8

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.7155. Min Act: -0.0000

Data Index: 733464 (The Pile)

Max Activating Token Index: 313

Text #9

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.7255. Min Act: -0.0000

Data Index: 1192478 (The Pile)

Max Activating Token Index: 264

Text #10

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.6945. Min Act: -0.0000

Data Index: 37441 (The Pile)

Max Activating Token Index: 622

Text #11

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.6945. Min Act: -0.0000

Data Index: 128147 (The Pile)

Max Activating Token Index: 133

Text #12

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.7690. Min Act: -0.0000

Data Index: 494771 (The Pile)

Max Activating Token Index: 616

Text #13

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.6842. Min Act: -0.0000

Data Index: 481053 (The Pile)

Max Activating Token Index: 338

Text #14

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.7013. Min Act: -0.0000

Data Index: 338327 (The Pile)

Max Activating Token Index: 153

Text #15

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.7494. Min Act: -0.0000

Data Index: 1711654 (The Pile)

Max Activating Token Index: 460

Text #16

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.6815. Min Act: -0.0000

Data Index: 1616403 (The Pile)

Max Activating Token Index: 81

Text #17

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.6983. Min Act: -0.0000

Data Index: 528377 (The Pile)

Max Activating Token Index: 133

Text #18

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.6436. Min Act: -0.0000

Data Index: 1215761 (The Pile)

Max Activating Token Index: 800

Text #19

Max Range: 0.9123. Min Range: -0.9123

Max Act: 0.6574. Min Act: -0.0000

Data Index: 244167 (The Pile)

Max Activating Token Index: 124