Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 188 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0693. Min Range: -1.0693

Max Act: 1.0559. Min Act: -0.0000

Data Index: 269188 (The Pile)

Max Activating Token Index: 552

Text #1

Max Range: 1.0693. Min Range: -1.0693

Max Act: 1.0693. Min Act: -0.0000

Data Index: 1119141 (The Pile)

Max Activating Token Index: 717

Text #2

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.9302. Min Act: -0.0000

Data Index: 1286922 (The Pile)

Max Activating Token Index: 541

Text #3

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.8011. Min Act: -0.0000

Data Index: 1056254 (The Pile)

Max Activating Token Index: 831

Text #4

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.8897. Min Act: -0.0000

Data Index: 1218483 (The Pile)

Max Activating Token Index: 556

Text #5

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.8450. Min Act: -0.0000

Data Index: 721806 (The Pile)

Max Activating Token Index: 644

Text #6

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.8505. Min Act: -0.0000

Data Index: 734149 (The Pile)

Max Activating Token Index: 1005

Text #7

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.8920. Min Act: -0.0000

Data Index: 1353209 (The Pile)

Max Activating Token Index: 275

Text #8

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.8429. Min Act: -0.0000

Data Index: 1774540 (The Pile)

Max Activating Token Index: 556

Text #9

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.8316. Min Act: -0.0000

Data Index: 403977 (The Pile)

Max Activating Token Index: 257

Text #10

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.8088. Min Act: 0.0000

Data Index: 1559092 (The Pile)

Max Activating Token Index: 258

Text #11

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.7686. Min Act: -0.0000

Data Index: 384912 (The Pile)

Max Activating Token Index: 1018

Text #12

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.7809. Min Act: -0.0000

Data Index: 61678 (The Pile)

Max Activating Token Index: 800

Text #13

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.8117. Min Act: -0.0000

Data Index: 754639 (The Pile)

Max Activating Token Index: 909

Text #14

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.7510. Min Act: -0.0000

Data Index: 1987411 (The Pile)

Max Activating Token Index: 621

Text #15

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.6902. Min Act: -0.0000

Data Index: 1082215 (The Pile)

Max Activating Token Index: 1009

Text #16

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.7455. Min Act: -0.0000

Data Index: 340709 (The Pile)

Max Activating Token Index: 692

Text #17

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.7162. Min Act: -0.0000

Data Index: 1202059 (The Pile)

Max Activating Token Index: 1002

Text #18

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.7186. Min Act: -0.0000

Data Index: 449165 (The Pile)

Max Activating Token Index: 744

Text #19

Max Range: 1.0693. Min Range: -1.0693

Max Act: 0.6719. Min Act: 0.0000

Data Index: 1611400 (The Pile)

Max Activating Token Index: 387