Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3893 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.5300. Min Act: -0.0000

Data Index: 1286789 (The Pile)

Max Activating Token Index: 740

Text #1

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.3917. Min Act: -0.0000

Data Index: 1962793 (The Pile)

Max Activating Token Index: 863

Text #2

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.3950. Min Act: -0.0000

Data Index: 187203 (The Pile)

Max Activating Token Index: 546

Text #3

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.3909. Min Act: -0.0000

Data Index: 1240031 (The Pile)

Max Activating Token Index: 472

Text #4

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.3063. Min Act: -0.0000

Data Index: 675581 (The Pile)

Max Activating Token Index: 106

Text #5

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.2094. Min Act: -0.0000

Data Index: 609723 (The Pile)

Max Activating Token Index: 216

Text #6

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.2869. Min Act: -0.0000

Data Index: 1906892 (The Pile)

Max Activating Token Index: 74

Text #7

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.2393. Min Act: -0.0000

Data Index: 572000 (The Pile)

Max Activating Token Index: 995

Text #8

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.1828. Min Act: -0.0000

Data Index: 345346 (The Pile)

Max Activating Token Index: 952

Text #9

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.2296. Min Act: -0.0000

Data Index: 1821511 (The Pile)

Max Activating Token Index: 152

Text #10

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.2979. Min Act: -0.0000

Data Index: 10156 (The Pile)

Max Activating Token Index: 973

Text #11

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.1222. Min Act: -0.0000

Data Index: 1663993 (The Pile)

Max Activating Token Index: 70

Text #12

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.2576. Min Act: -0.0000

Data Index: 972274 (The Pile)

Max Activating Token Index: 332

Text #13

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.2436. Min Act: -0.0000

Data Index: 1314151 (The Pile)

Max Activating Token Index: 336

Text #14

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.1615. Min Act: -0.0000

Data Index: 217662 (The Pile)

Max Activating Token Index: 761

Text #15

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.0819. Min Act: -0.0000

Data Index: 957429 (The Pile)

Max Activating Token Index: 884

Text #16

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.1172. Min Act: -0.0000

Data Index: 481634 (The Pile)

Max Activating Token Index: 7

Text #17

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.1506. Min Act: -0.0000

Data Index: 862329 (The Pile)

Max Activating Token Index: 164

Text #18

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.3500. Min Act: -0.0000

Data Index: 1965305 (The Pile)

Max Activating Token Index: 960

Text #19

Max Range: 1.5300. Min Range: -1.5300

Max Act: 1.0730. Min Act: -0.0000

Data Index: 251971 (The Pile)

Max Activating Token Index: 185