Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 252 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.5299. Min Range: -3.5299

Max Act: 3.5299. Min Act: -0.0000

Data Index: 520273 (The Pile)

Max Activating Token Index: 117

Text #1

Max Range: 3.5299. Min Range: -3.5299

Max Act: 3.2224. Min Act: -0.0000

Data Index: 188090 (The Pile)

Max Activating Token Index: 108

Text #2

Max Range: 3.5299. Min Range: -3.5299

Max Act: 3.2200. Min Act: -0.0000

Data Index: 1983171 (The Pile)

Max Activating Token Index: 46

Text #3

Max Range: 3.5299. Min Range: -3.5299

Max Act: 3.4981. Min Act: -0.0000

Data Index: 514642 (The Pile)

Max Activating Token Index: 576

Text #4

Max Range: 3.5299. Min Range: -3.5299

Max Act: 3.3515. Min Act: -0.0000

Data Index: 1361362 (The Pile)

Max Activating Token Index: 124

Text #5

Max Range: 3.5299. Min Range: -3.5299

Max Act: 3.1214. Min Act: -0.0000

Data Index: 1499149 (The Pile)

Max Activating Token Index: 466

Text #6

Max Range: 3.5299. Min Range: -3.5299

Max Act: 2.6962. Min Act: -0.0000

Data Index: 193841 (The Pile)

Max Activating Token Index: 58

Text #7

Max Range: 3.5299. Min Range: -3.5299

Max Act: 2.9642. Min Act: -0.0000

Data Index: 1879722 (The Pile)

Max Activating Token Index: 249

Text #8

Max Range: 3.5299. Min Range: -3.5299

Max Act: 2.6810. Min Act: -0.0000

Data Index: 375041 (The Pile)

Max Activating Token Index: 723

Text #9

Max Range: 3.5299. Min Range: -3.5299

Max Act: 2.8224. Min Act: -0.0000

Data Index: 580364 (The Pile)

Max Activating Token Index: 206

Text #10

Max Range: 3.5299. Min Range: -3.5299

Max Act: 2.7931. Min Act: -0.0000

Data Index: 935090 (The Pile)

Max Activating Token Index: 32

Text #11

Max Range: 3.5299. Min Range: -3.5299

Max Act: 3.1483. Min Act: -0.0000

Data Index: 1229182 (The Pile)

Max Activating Token Index: 862

Text #12

Max Range: 3.5299. Min Range: -3.5299

Max Act: 2.6871. Min Act: -0.0000

Data Index: 90645 (The Pile)

Max Activating Token Index: 320

Text #13

Max Range: 3.5299. Min Range: -3.5299

Max Act: 2.6643. Min Act: -0.0000

Data Index: 1345602 (The Pile)

Max Activating Token Index: 460

Text #14

Max Range: 3.5299. Min Range: -3.5299

Max Act: 2.4298. Min Act: -0.0000

Data Index: 108197 (The Pile)

Max Activating Token Index: 209

Text #15

Max Range: 3.5299. Min Range: -3.5299

Max Act: 2.5364. Min Act: -0.0000

Data Index: 1752625 (The Pile)

Max Activating Token Index: 161

Text #16

Max Range: 3.5299. Min Range: -3.5299

Max Act: 2.2826. Min Act: -0.0000

Data Index: 1827044 (The Pile)

Max Activating Token Index: 387

Text #17

Max Range: 3.5299. Min Range: -3.5299

Max Act: 2.2169. Min Act: -0.0000

Data Index: 290670 (The Pile)

Max Activating Token Index: 811

Text #18

Max Range: 3.5299. Min Range: -3.5299

Max Act: 2.5949. Min Act: -0.0000

Data Index: 522798 (The Pile)

Max Activating Token Index: 824

Text #19

Max Range: 3.5299. Min Range: -3.5299

Max Act: 2.1501. Min Act: -0.0000

Data Index: 908507 (The Pile)

Max Activating Token Index: 593