Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 5108 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.6984. Min Act: -0.0000

Data Index: 807808 (The Pile)

Max Activating Token Index: 748

Text #1

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.7286. Min Act: -0.0000

Data Index: 916203 (The Pile)

Max Activating Token Index: 178

Text #2

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.6881. Min Act: -0.0000

Data Index: 1582870 (The Pile)

Max Activating Token Index: 104

Text #3

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.6537. Min Act: -0.0000

Data Index: 403904 (The Pile)

Max Activating Token Index: 98

Text #4

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.6391. Min Act: -0.0000

Data Index: 1377764 (The Pile)

Max Activating Token Index: 171

Text #5

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.6833. Min Act: -0.0000

Data Index: 1363837 (The Pile)

Max Activating Token Index: 56

Text #6

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.6687. Min Act: -0.0000

Data Index: 1663380 (The Pile)

Max Activating Token Index: 342

Text #7

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.6453. Min Act: -0.0000

Data Index: 181834 (The Pile)

Max Activating Token Index: 220

Text #8

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.6375. Min Act: -0.0000

Data Index: 1794923 (The Pile)

Max Activating Token Index: 584

Text #9

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.6338. Min Act: -0.0000

Data Index: 245101 (The Pile)

Max Activating Token Index: 131

Text #10

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.6275. Min Act: -0.0000

Data Index: 1032238 (The Pile)

Max Activating Token Index: 93

Text #11

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.5669. Min Act: -0.0000

Data Index: 165310 (The Pile)

Max Activating Token Index: 188

Text #12

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.5421. Min Act: -0.0000

Data Index: 1742474 (The Pile)

Max Activating Token Index: 931

Text #13

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.5712. Min Act: -0.0000

Data Index: 325420 (The Pile)

Max Activating Token Index: 720

Text #14

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.6037. Min Act: -0.0000

Data Index: 1169407 (The Pile)

Max Activating Token Index: 44

Text #15

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.5767. Min Act: -0.0000

Data Index: 468711 (The Pile)

Max Activating Token Index: 727

Text #16

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.5554. Min Act: -0.0000

Data Index: 277021 (The Pile)

Max Activating Token Index: 120

Text #17

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.5173. Min Act: -0.0000

Data Index: 958215 (The Pile)

Max Activating Token Index: 633

Text #18

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.5725. Min Act: -0.0000

Data Index: 405935 (The Pile)

Max Activating Token Index: 307

Text #19

Max Range: 0.7286. Min Range: -0.7286

Max Act: 0.5308. Min Act: -0.0000

Data Index: 564375 (The Pile)

Max Activating Token Index: 1014