Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2203 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.3759. Min Act: -0.0001

Data Index: 58039 (The Pile)

Max Activating Token Index: 285

Text #1

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.3244. Min Act: -0.0000

Data Index: 963076 (The Pile)

Max Activating Token Index: 65

Text #2

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.3414. Min Act: -0.0000

Data Index: 284386 (The Pile)

Max Activating Token Index: 810

Text #3

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.3409. Min Act: -0.0000

Data Index: 1114593 (The Pile)

Max Activating Token Index: 483

Text #4

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.3481. Min Act: -0.0000

Data Index: 1075729 (The Pile)

Max Activating Token Index: 813

Text #5

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.3248. Min Act: -0.0000

Data Index: 976056 (The Pile)

Max Activating Token Index: 144

Text #6

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.3317. Min Act: -0.0000

Data Index: 632508 (The Pile)

Max Activating Token Index: 944

Text #7

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.3063. Min Act: -0.0000

Data Index: 46627 (The Pile)

Max Activating Token Index: 301

Text #8

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.3194. Min Act: -0.0000

Data Index: 618298 (The Pile)

Max Activating Token Index: 300

Text #9

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.2991. Min Act: -0.0000

Data Index: 1904391 (The Pile)

Max Activating Token Index: 23

Text #10

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.2768. Min Act: -0.0000

Data Index: 48017 (The Pile)

Max Activating Token Index: 982

Text #11

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.3055. Min Act: -0.0000

Data Index: 364582 (The Pile)

Max Activating Token Index: 393

Text #12

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.3133. Min Act: -0.0000

Data Index: 1050614 (The Pile)

Max Activating Token Index: 710

Text #13

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.2930. Min Act: -0.0000

Data Index: 952443 (The Pile)

Max Activating Token Index: 153

Text #14

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.3035. Min Act: -0.0000

Data Index: 688700 (The Pile)

Max Activating Token Index: 304

Text #15

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.3172. Min Act: -0.0000

Data Index: 1100612 (The Pile)

Max Activating Token Index: 627

Text #16

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.2885. Min Act: -0.0000

Data Index: 601370 (The Pile)

Max Activating Token Index: 874

Text #17

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.2796. Min Act: -0.0000

Data Index: 1205068 (The Pile)

Max Activating Token Index: 793

Text #18

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.2964. Min Act: -0.0000

Data Index: 1099803 (The Pile)

Max Activating Token Index: 156

Text #19

Max Range: 0.3759. Min Range: -0.3759

Max Act: 0.2843. Min Act: -0.0000

Data Index: 544721 (The Pile)

Max Activating Token Index: 59