Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1363 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.2253. Min Range: -1.2253

Max Act: 1.2253. Min Act: -0.0000

Data Index: 1433205 (The Pile)

Max Activating Token Index: 557

Text #1

Max Range: 1.2253. Min Range: -1.2253

Max Act: 1.1916. Min Act: 0.0000

Data Index: 163916 (The Pile)

Max Activating Token Index: 816

Text #2

Max Range: 1.2253. Min Range: -1.2253

Max Act: 1.1556. Min Act: -0.0000

Data Index: 1418826 (The Pile)

Max Activating Token Index: 429

Text #3

Max Range: 1.2253. Min Range: -1.2253

Max Act: 1.1049. Min Act: -0.0000

Data Index: 1641520 (The Pile)

Max Activating Token Index: 598

Text #4

Max Range: 1.2253. Min Range: -1.2253

Max Act: 0.9646. Min Act: -0.0000

Data Index: 990293 (The Pile)

Max Activating Token Index: 574

Text #5

Max Range: 1.2253. Min Range: -1.2253

Max Act: 1.0164. Min Act: -0.0000

Data Index: 1281335 (The Pile)

Max Activating Token Index: 744

Text #6

Max Range: 1.2253. Min Range: -1.2253

Max Act: 0.8970. Min Act: -0.0000

Data Index: 1199311 (The Pile)

Max Activating Token Index: 969

Text #7

Max Range: 1.2253. Min Range: -1.2253

Max Act: 0.8973. Min Act: -0.0000

Data Index: 476119 (The Pile)

Max Activating Token Index: 767

Text #8

Max Range: 1.2253. Min Range: -1.2253

Max Act: 0.9504. Min Act: -0.0000

Data Index: 565266 (The Pile)

Max Activating Token Index: 670

Text #9

Max Range: 1.2253. Min Range: -1.2253

Max Act: 0.8743. Min Act: -0.0000

Data Index: 641856 (The Pile)

Max Activating Token Index: 448

Text #10

Max Range: 1.2253. Min Range: -1.2253

Max Act: 0.9010. Min Act: -0.0000

Data Index: 505931 (The Pile)

Max Activating Token Index: 780

Text #11

Max Range: 1.2253. Min Range: -1.2253

Max Act: 0.8204. Min Act: -0.0000

Data Index: 1076370 (The Pile)

Max Activating Token Index: 592

Text #12

Max Range: 1.2253. Min Range: -1.2253

Max Act: 0.8000. Min Act: -0.0000

Data Index: 1533357 (The Pile)

Max Activating Token Index: 671

Text #13

Max Range: 1.2253. Min Range: -1.2253

Max Act: 0.8278. Min Act: -0.0000

Data Index: 249897 (The Pile)

Max Activating Token Index: 434

Text #14

Max Range: 1.2253. Min Range: -1.2253

Max Act: 0.8174. Min Act: -0.0000

Data Index: 1761795 (The Pile)

Max Activating Token Index: 338

Text #15

Max Range: 1.2253. Min Range: -1.2253

Max Act: 0.8418. Min Act: -0.0000

Data Index: 763325 (The Pile)

Max Activating Token Index: 269

Text #16

Max Range: 1.2253. Min Range: -1.2253

Max Act: 0.8126. Min Act: -0.0000

Data Index: 554482 (The Pile)

Max Activating Token Index: 474

Text #17

Max Range: 1.2253. Min Range: -1.2253

Max Act: 0.6796. Min Act: -0.0000

Data Index: 1056421 (The Pile)

Max Activating Token Index: 855

Text #18

Max Range: 1.2253. Min Range: -1.2253

Max Act: 0.7969. Min Act: -0.0000

Data Index: 320567 (The Pile)

Max Activating Token Index: 63

Text #19

Max Range: 1.2253. Min Range: -1.2253

Max Act: 0.8104. Min Act: -0.0000

Data Index: 684320 (The Pile)

Max Activating Token Index: 550