Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3897 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.1166. Min Act: -0.0000

Data Index: 1591411 (The Pile)

Max Activating Token Index: 407

Text #1

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.1218. Min Act: -0.0000

Data Index: 970771 (The Pile)

Max Activating Token Index: 312

Text #2

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.1136. Min Act: -0.0000

Data Index: 1880566 (The Pile)

Max Activating Token Index: 984

Text #3

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.1064. Min Act: -0.0000

Data Index: 1741363 (The Pile)

Max Activating Token Index: 296

Text #4

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.1088. Min Act: -0.0000

Data Index: 459929 (The Pile)

Max Activating Token Index: 1004

Text #5

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.0979. Min Act: -0.0000

Data Index: 855154 (The Pile)

Max Activating Token Index: 854

Text #6

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.1040. Min Act: -0.0000

Data Index: 1372239 (The Pile)

Max Activating Token Index: 430

Text #7

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.1078. Min Act: -0.0000

Data Index: 1107221 (The Pile)

Max Activating Token Index: 777

Text #8

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.0935. Min Act: -0.0000

Data Index: 772401 (The Pile)

Max Activating Token Index: 822

Text #9

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.1025. Min Act: -0.0000

Data Index: 1230872 (The Pile)

Max Activating Token Index: 546

Text #10

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.0988. Min Act: -0.0000

Data Index: 1426202 (The Pile)

Max Activating Token Index: 596

Text #11

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.0992. Min Act: -0.0000

Data Index: 1076419 (The Pile)

Max Activating Token Index: 688

Text #12

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.0912. Min Act: -0.0000

Data Index: 311544 (The Pile)

Max Activating Token Index: 339

Text #13

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.0845. Min Act: -0.0000

Data Index: 903838 (The Pile)

Max Activating Token Index: 1016

Text #14

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.0936. Min Act: -0.0000

Data Index: 1766369 (The Pile)

Max Activating Token Index: 39

Text #15

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.0958. Min Act: -0.0000

Data Index: 504225 (The Pile)

Max Activating Token Index: 1009

Text #16

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.0819. Min Act: -0.0000

Data Index: 1185945 (The Pile)

Max Activating Token Index: 929

Text #17

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.0818. Min Act: -0.0000

Data Index: 1817524 (The Pile)

Max Activating Token Index: 979

Text #18

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.0780. Min Act: -0.0000

Data Index: 1305525 (The Pile)

Max Activating Token Index: 178

Text #19

Max Range: 0.1218. Min Range: -0.1218

Max Act: 0.0868. Min Act: -0.0000

Data Index: 984485 (The Pile)

Max Activating Token Index: 965