Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2269 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.6103. Min Range: -1.6103

Max Act: 1.6103. Min Act: -0.0000

Data Index: 1696023 (The Pile)

Max Activating Token Index: 436

Text #1

Max Range: 1.6103. Min Range: -1.6103

Max Act: 1.5363. Min Act: -0.0000

Data Index: 1714207 (The Pile)

Max Activating Token Index: 489

Text #2

Max Range: 1.6103. Min Range: -1.6103

Max Act: 1.3460. Min Act: -0.0000

Data Index: 468923 (The Pile)

Max Activating Token Index: 850

Text #3

Max Range: 1.6103. Min Range: -1.6103

Max Act: 1.1028. Min Act: -0.0000

Data Index: 1991983 (The Pile)

Max Activating Token Index: 843

Text #4

Max Range: 1.6103. Min Range: -1.6103

Max Act: 1.0171. Min Act: -0.0000

Data Index: 1678045 (The Pile)

Max Activating Token Index: 725

Text #5

Max Range: 1.6103. Min Range: -1.6103

Max Act: 1.0043. Min Act: -0.0000

Data Index: 1094419 (The Pile)

Max Activating Token Index: 332

Text #6

Max Range: 1.6103. Min Range: -1.6103

Max Act: 1.0307. Min Act: -0.0000

Data Index: 665969 (The Pile)

Max Activating Token Index: 102

Text #7

Max Range: 1.6103. Min Range: -1.6103

Max Act: 1.3447. Min Act: -0.0000

Data Index: 1779311 (The Pile)

Max Activating Token Index: 926

Text #8

Max Range: 1.6103. Min Range: -1.6103

Max Act: 1.0777. Min Act: -0.0000

Data Index: 1015181 (The Pile)

Max Activating Token Index: 927

Text #9

Max Range: 1.6103. Min Range: -1.6103

Max Act: 0.9399. Min Act: -0.0000

Data Index: 1476026 (The Pile)

Max Activating Token Index: 694

Text #10

Max Range: 1.6103. Min Range: -1.6103

Max Act: 0.9577. Min Act: -0.0000

Data Index: 1733627 (The Pile)

Max Activating Token Index: 556

Text #11

Max Range: 1.6103. Min Range: -1.6103

Max Act: 1.1279. Min Act: -0.0000

Data Index: 493271 (The Pile)

Max Activating Token Index: 487

Text #12

Max Range: 1.6103. Min Range: -1.6103

Max Act: 0.9389. Min Act: -0.0000

Data Index: 1230785 (The Pile)

Max Activating Token Index: 1012

Text #13

Max Range: 1.6103. Min Range: -1.6103

Max Act: 0.9040. Min Act: -0.0000

Data Index: 1977165 (The Pile)

Max Activating Token Index: 377

Text #14

Max Range: 1.6103. Min Range: -1.6103

Max Act: 0.9345. Min Act: -0.0000

Data Index: 598239 (The Pile)

Max Activating Token Index: 597

Text #15

Max Range: 1.6103. Min Range: -1.6103

Max Act: 0.9723. Min Act: -0.0000

Data Index: 1074085 (The Pile)

Max Activating Token Index: 1020

Text #16

Max Range: 1.6103. Min Range: -1.6103

Max Act: 0.7410. Min Act: -0.0000

Data Index: 821978 (The Pile)

Max Activating Token Index: 734

Text #17

Max Range: 1.6103. Min Range: -1.6103

Max Act: 0.8837. Min Act: -0.0000

Data Index: 1773382 (The Pile)

Max Activating Token Index: 119

Text #18

Max Range: 1.6103. Min Range: -1.6103

Max Act: 0.8291. Min Act: -0.0000

Data Index: 1533083 (The Pile)

Max Activating Token Index: 993

Text #19

Max Range: 1.6103. Min Range: -1.6103

Max Act: 0.9212. Min Act: -0.0000

Data Index: 488832 (The Pile)

Max Activating Token Index: 914