Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3453 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.8039. Min Range: -3.8039

Max Act: 3.8039. Min Act: -0.0000

Data Index: 202623 (The Pile)

Max Activating Token Index: 245

Text #1

Max Range: 3.8039. Min Range: -3.8039

Max Act: 3.7019. Min Act: -0.0000

Data Index: 1309345 (The Pile)

Max Activating Token Index: 205

Text #2

Max Range: 3.8039. Min Range: -3.8039

Max Act: 3.2711. Min Act: -0.0000

Data Index: 310594 (The Pile)

Max Activating Token Index: 546

Text #3

Max Range: 3.8039. Min Range: -3.8039

Max Act: 2.9588. Min Act: -0.0000

Data Index: 1362090 (The Pile)

Max Activating Token Index: 559

Text #4

Max Range: 3.8039. Min Range: -3.8039

Max Act: 2.9344. Min Act: -0.0000

Data Index: 1110120 (The Pile)

Max Activating Token Index: 428

Text #5

Max Range: 3.8039. Min Range: -3.8039

Max Act: 3.0321. Min Act: -0.0000

Data Index: 1429403 (The Pile)

Max Activating Token Index: 425

Text #6

Max Range: 3.8039. Min Range: -3.8039

Max Act: 2.8405. Min Act: -0.0000

Data Index: 1591212 (The Pile)

Max Activating Token Index: 890

Text #7

Max Range: 3.8039. Min Range: -3.8039

Max Act: 2.8874. Min Act: -0.0000

Data Index: 987853 (The Pile)

Max Activating Token Index: 801

Text #8

Max Range: 3.8039. Min Range: -3.8039

Max Act: 2.8805. Min Act: -0.0000

Data Index: 1490729 (The Pile)

Max Activating Token Index: 40

Text #9

Max Range: 3.8039. Min Range: -3.8039

Max Act: 2.6449. Min Act: -0.0000

Data Index: 330418 (The Pile)

Max Activating Token Index: 369

Text #10

Max Range: 3.8039. Min Range: -3.8039

Max Act: 2.7358. Min Act: -0.0000

Data Index: 1129335 (The Pile)

Max Activating Token Index: 318

Text #11

Max Range: 3.8039. Min Range: -3.8039

Max Act: 2.6991. Min Act: -0.0000

Data Index: 1937323 (The Pile)

Max Activating Token Index: 754

Text #12

Max Range: 3.8039. Min Range: -3.8039

Max Act: 2.8348. Min Act: -0.0000

Data Index: 1360091 (The Pile)

Max Activating Token Index: 469

Text #13

Max Range: 3.8039. Min Range: -3.8039

Max Act: 2.6781. Min Act: -0.0000

Data Index: 386112 (The Pile)

Max Activating Token Index: 753

Text #14

Max Range: 3.8039. Min Range: -3.8039

Max Act: 2.6494. Min Act: -0.0000

Data Index: 1422244 (The Pile)

Max Activating Token Index: 661

Text #15

Max Range: 3.8039. Min Range: -3.8039

Max Act: 2.4551. Min Act: -0.0000

Data Index: 1792612 (The Pile)

Max Activating Token Index: 555

Text #16

Max Range: 3.8039. Min Range: -3.8039

Max Act: 2.4334. Min Act: -0.0000

Data Index: 21701 (The Pile)

Max Activating Token Index: 737

Text #17

Max Range: 3.8039. Min Range: -3.8039

Max Act: 2.5006. Min Act: -0.0000

Data Index: 1485773 (The Pile)

Max Activating Token Index: 665

Text #18

Max Range: 3.8039. Min Range: -3.8039

Max Act: 2.5059. Min Act: -0.0000

Data Index: 589496 (The Pile)

Max Activating Token Index: 417

Text #19

Max Range: 3.8039. Min Range: -3.8039

Max Act: 2.5142. Min Act: -0.0000

Data Index: 1552164 (The Pile)

Max Activating Token Index: 61