Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2809 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1344. Min Range: -1.1344

Max Act: 1.1229. Min Act: -0.0000

Data Index: 1652678 (The Pile)

Max Activating Token Index: 433

Text #1

Max Range: 1.1344. Min Range: -1.1344

Max Act: 1.0070. Min Act: -0.0000

Data Index: 260524 (The Pile)

Max Activating Token Index: 752

Text #2

Max Range: 1.1344. Min Range: -1.1344

Max Act: 1.1344. Min Act: -0.0000

Data Index: 874626 (The Pile)

Max Activating Token Index: 37

Text #3

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.9851. Min Act: -0.0000

Data Index: 1591737 (The Pile)

Max Activating Token Index: 499

Text #4

Max Range: 1.1344. Min Range: -1.1344

Max Act: 1.0355. Min Act: -0.0000

Data Index: 93336 (The Pile)

Max Activating Token Index: 86

Text #5

Max Range: 1.1344. Min Range: -1.1344

Max Act: 1.0142. Min Act: -0.0000

Data Index: 1874662 (The Pile)

Max Activating Token Index: 876

Text #6

Max Range: 1.1344. Min Range: -1.1344

Max Act: 1.1022. Min Act: -0.0000

Data Index: 1223720 (The Pile)

Max Activating Token Index: 174

Text #7

Max Range: 1.1344. Min Range: -1.1344

Max Act: 1.0078. Min Act: -0.0000

Data Index: 1433950 (The Pile)

Max Activating Token Index: 864

Text #8

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.9922. Min Act: -0.0000

Data Index: 1509374 (The Pile)

Max Activating Token Index: 671

Text #9

Max Range: 1.1344. Min Range: -1.1344

Max Act: 1.0401. Min Act: -0.0000

Data Index: 1150224 (The Pile)

Max Activating Token Index: 57

Text #10

Max Range: 1.1344. Min Range: -1.1344

Max Act: 1.0185. Min Act: -0.0000

Data Index: 711145 (The Pile)

Max Activating Token Index: 449

Text #11

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.9363. Min Act: -0.0000

Data Index: 530252 (The Pile)

Max Activating Token Index: 119

Text #12

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.9326. Min Act: -0.0000

Data Index: 1791238 (The Pile)

Max Activating Token Index: 64

Text #13

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.8981. Min Act: -0.0000

Data Index: 1130469 (The Pile)

Max Activating Token Index: 228

Text #14

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.9243. Min Act: -0.0000

Data Index: 743319 (The Pile)

Max Activating Token Index: 969

Text #15

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.9777. Min Act: -0.0000

Data Index: 856679 (The Pile)

Max Activating Token Index: 270

Text #16

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.9406. Min Act: -0.0000

Data Index: 408949 (The Pile)

Max Activating Token Index: 627

Text #17

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.9382. Min Act: -0.0000

Data Index: 1435847 (The Pile)

Max Activating Token Index: 895

Text #18

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.9433. Min Act: -0.0000

Data Index: 99792 (The Pile)

Max Activating Token Index: 490

Text #19

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.9838. Min Act: -0.0000

Data Index: 1492632 (The Pile)

Max Activating Token Index: 700