Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4211 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.3277. Min Range: -1.3277

Max Act: 1.3277. Min Act: -0.0000

Data Index: 415942 (The Pile)

Max Activating Token Index: 766

Text #1

Max Range: 1.3277. Min Range: -1.3277

Max Act: 1.2816. Min Act: -0.0000

Data Index: 1174636 (The Pile)

Max Activating Token Index: 504

Text #2

Max Range: 1.3277. Min Range: -1.3277

Max Act: 1.0882. Min Act: -0.0000

Data Index: 1553213 (The Pile)

Max Activating Token Index: 328

Text #3

Max Range: 1.3277. Min Range: -1.3277

Max Act: 1.0001. Min Act: -0.0000

Data Index: 807998 (The Pile)

Max Activating Token Index: 759

Text #4

Max Range: 1.3277. Min Range: -1.3277

Max Act: 1.0630. Min Act: -0.0000

Data Index: 1990939 (The Pile)

Max Activating Token Index: 1000

Text #5

Max Range: 1.3277. Min Range: -1.3277

Max Act: 1.0280. Min Act: -0.0000

Data Index: 896224 (The Pile)

Max Activating Token Index: 398

Text #6

Max Range: 1.3277. Min Range: -1.3277

Max Act: 1.0074. Min Act: -0.0000

Data Index: 423291 (The Pile)

Max Activating Token Index: 878

Text #7

Max Range: 1.3277. Min Range: -1.3277

Max Act: 1.0935. Min Act: -0.0000

Data Index: 72124 (The Pile)

Max Activating Token Index: 113

Text #8

Max Range: 1.3277. Min Range: -1.3277

Max Act: 1.0734. Min Act: -0.0000

Data Index: 1377352 (The Pile)

Max Activating Token Index: 337

Text #9

Max Range: 1.3277. Min Range: -1.3277

Max Act: 1.0611. Min Act: -0.0000

Data Index: 1303793 (The Pile)

Max Activating Token Index: 717

Text #10

Max Range: 1.3277. Min Range: -1.3277

Max Act: 1.0056. Min Act: -0.0000

Data Index: 1491615 (The Pile)

Max Activating Token Index: 822

Text #11

Max Range: 1.3277. Min Range: -1.3277

Max Act: 1.0077. Min Act: -0.0000

Data Index: 730151 (The Pile)

Max Activating Token Index: 275

Text #12

Max Range: 1.3277. Min Range: -1.3277

Max Act: 1.0368. Min Act: -0.0000

Data Index: 1670961 (The Pile)

Max Activating Token Index: 580

Text #13

Max Range: 1.3277. Min Range: -1.3277

Max Act: 0.9367. Min Act: -0.0000

Data Index: 1118627 (The Pile)

Max Activating Token Index: 696

Text #14

Max Range: 1.3277. Min Range: -1.3277

Max Act: 0.8680. Min Act: -0.0000

Data Index: 1506811 (The Pile)

Max Activating Token Index: 701

Text #15

Max Range: 1.3277. Min Range: -1.3277

Max Act: 0.9382. Min Act: -0.0000

Data Index: 1016659 (The Pile)

Max Activating Token Index: 271

Text #16

Max Range: 1.3277. Min Range: -1.3277

Max Act: 1.0385. Min Act: -0.0000

Data Index: 1894677 (The Pile)

Max Activating Token Index: 765

Text #17

Max Range: 1.3277. Min Range: -1.3277

Max Act: 0.9688. Min Act: -0.0000

Data Index: 969099 (The Pile)

Max Activating Token Index: 811

Text #18

Max Range: 1.3277. Min Range: -1.3277

Max Act: 0.9216. Min Act: -0.0000

Data Index: 612612 (The Pile)

Max Activating Token Index: 456

Text #19

Max Range: 1.3277. Min Range: -1.3277

Max Act: 0.8086. Min Act: -0.0000

Data Index: 579499 (The Pile)

Max Activating Token Index: 190