Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1816 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.7167. Min Range: -3.7167

Max Act: 3.7167. Min Act: -0.0000

Data Index: 1822070 (The Pile)

Max Activating Token Index: 326

Text #1

Max Range: 3.7167. Min Range: -3.7167

Max Act: 3.6892. Min Act: -0.0000

Data Index: 725865 (The Pile)

Max Activating Token Index: 177

Text #2

Max Range: 3.7167. Min Range: -3.7167

Max Act: 3.4304. Min Act: -0.0000

Data Index: 14326 (The Pile)

Max Activating Token Index: 496

Text #3

Max Range: 3.7167. Min Range: -3.7167

Max Act: 3.5496. Min Act: -0.0000

Data Index: 1553115 (The Pile)

Max Activating Token Index: 115

Text #4

Max Range: 3.7167. Min Range: -3.7167

Max Act: 3.2149. Min Act: -0.0000

Data Index: 1852542 (The Pile)

Max Activating Token Index: 205

Text #5

Max Range: 3.7167. Min Range: -3.7167

Max Act: 3.5239. Min Act: -0.0000

Data Index: 704697 (The Pile)

Max Activating Token Index: 842

Text #6

Max Range: 3.7167. Min Range: -3.7167

Max Act: 3.0272. Min Act: -0.0000

Data Index: 1440185 (The Pile)

Max Activating Token Index: 170

Text #7

Max Range: 3.7167. Min Range: -3.7167

Max Act: 3.1626. Min Act: -0.0000

Data Index: 1440250 (The Pile)

Max Activating Token Index: 866

Text #8

Max Range: 3.7167. Min Range: -3.7167

Max Act: 3.6913. Min Act: -0.0000

Data Index: 1114207 (The Pile)

Max Activating Token Index: 938

Text #9

Max Range: 3.7167. Min Range: -3.7167

Max Act: 3.1584. Min Act: -0.0000

Data Index: 1280975 (The Pile)

Max Activating Token Index: 925

Text #10

Max Range: 3.7167. Min Range: -3.7167

Max Act: 2.9448. Min Act: -0.0000

Data Index: 714100 (The Pile)

Max Activating Token Index: 854

Text #11

Max Range: 3.7167. Min Range: -3.7167

Max Act: 2.8535. Min Act: -0.0000

Data Index: 305339 (The Pile)

Max Activating Token Index: 619

Text #12

Max Range: 3.7167. Min Range: -3.7167

Max Act: 2.7246. Min Act: -0.0000

Data Index: 59526 (The Pile)

Max Activating Token Index: 756

Text #13

Max Range: 3.7167. Min Range: -3.7167

Max Act: 2.4573. Min Act: -0.0000

Data Index: 617897 (The Pile)

Max Activating Token Index: 718

Text #14

Max Range: 3.7167. Min Range: -3.7167

Max Act: 2.5159. Min Act: -0.0000

Data Index: 1494041 (The Pile)

Max Activating Token Index: 351

Text #15

Max Range: 3.7167. Min Range: -3.7167

Max Act: 2.7476. Min Act: -0.0000

Data Index: 1913571 (The Pile)

Max Activating Token Index: 815

Text #16

Max Range: 3.7167. Min Range: -3.7167

Max Act: 2.4831. Min Act: -0.0000

Data Index: 1566429 (The Pile)

Max Activating Token Index: 447

Text #17

Max Range: 3.7167. Min Range: -3.7167

Max Act: 2.9022. Min Act: -0.0000

Data Index: 1760005 (The Pile)

Max Activating Token Index: 91

Text #18

Max Range: 3.7167. Min Range: -3.7167

Max Act: 2.7002. Min Act: -0.0000

Data Index: 663452 (The Pile)

Max Activating Token Index: 275

Text #19

Max Range: 3.7167. Min Range: -3.7167

Max Act: 2.4416. Min Act: -0.0000

Data Index: 1288470 (The Pile)

Max Activating Token Index: 377