Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3810 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.5276. Min Act: 0.0000

Data Index: 702067 (The Pile)

Max Activating Token Index: 771

Text #1

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.5279. Min Act: -0.0000

Data Index: 997322 (The Pile)

Max Activating Token Index: 78

Text #2

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.5333. Min Act: -0.0000

Data Index: 427044 (The Pile)

Max Activating Token Index: 996

Text #3

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.5296. Min Act: 0.0000

Data Index: 1141869 (The Pile)

Max Activating Token Index: 298

Text #4

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.5188. Min Act: -0.0000

Data Index: 923318 (The Pile)

Max Activating Token Index: 354

Text #5

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.4978. Min Act: 0.0000

Data Index: 48337 (The Pile)

Max Activating Token Index: 464

Text #6

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.5016. Min Act: -0.0000

Data Index: 1860414 (The Pile)

Max Activating Token Index: 692

Text #7

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.4743. Min Act: -0.0000

Data Index: 1084741 (The Pile)

Max Activating Token Index: 894

Text #8

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.4822. Min Act: -0.0000

Data Index: 568983 (The Pile)

Max Activating Token Index: 108

Text #9

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.4825. Min Act: -0.0000

Data Index: 182661 (The Pile)

Max Activating Token Index: 908

Text #10

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.4799. Min Act: 0.0000

Data Index: 216438 (The Pile)

Max Activating Token Index: 581

Text #11

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.4448. Min Act: -0.0000

Data Index: 1686884 (The Pile)

Max Activating Token Index: 782

Text #12

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.5395. Min Act: 0.0000

Data Index: 1233405 (The Pile)

Max Activating Token Index: 763

Text #13

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.4401. Min Act: 0.0000

Data Index: 1787525 (The Pile)

Max Activating Token Index: 504

Text #14

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.4641. Min Act: -0.0000

Data Index: 831973 (The Pile)

Max Activating Token Index: 633

Text #15

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.4571. Min Act: -0.0000

Data Index: 1911304 (The Pile)

Max Activating Token Index: 280

Text #16

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.4659. Min Act: -0.0000

Data Index: 1183751 (The Pile)

Max Activating Token Index: 715

Text #17

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.4447. Min Act: 0.0000

Data Index: 1799957 (The Pile)

Max Activating Token Index: 135

Text #18

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.4373. Min Act: 0.0000

Data Index: 1103974 (The Pile)

Max Activating Token Index: 428

Text #19

Max Range: 0.5395. Min Range: -0.5395

Max Act: 0.4259. Min Act: 0.0000

Data Index: 50083 (The Pile)

Max Activating Token Index: 859