Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 572 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.8848. Min Act: -0.0000

Data Index: 318193 (The Pile)

Max Activating Token Index: 510

Text #1

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.7984. Min Act: -0.0000

Data Index: 450536 (The Pile)

Max Activating Token Index: 356

Text #2

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.6927. Min Act: -0.0000

Data Index: 7868 (The Pile)

Max Activating Token Index: 244

Text #3

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.7128. Min Act: -0.0000

Data Index: 756985 (The Pile)

Max Activating Token Index: 446

Text #4

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.7069. Min Act: -0.0000

Data Index: 1107289 (The Pile)

Max Activating Token Index: 244

Text #5

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.7323. Min Act: -0.0000

Data Index: 376824 (The Pile)

Max Activating Token Index: 637

Text #6

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.6964. Min Act: -0.0000

Data Index: 727263 (The Pile)

Max Activating Token Index: 41

Text #7

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.7397. Min Act: -0.0000

Data Index: 1382197 (The Pile)

Max Activating Token Index: 888

Text #8

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.7575. Min Act: -0.0000

Data Index: 1508319 (The Pile)

Max Activating Token Index: 982

Text #9

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.6663. Min Act: -0.0000

Data Index: 374696 (The Pile)

Max Activating Token Index: 731

Text #10

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.6786. Min Act: -0.0000

Data Index: 1620876 (The Pile)

Max Activating Token Index: 145

Text #11

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.6972. Min Act: -0.0000

Data Index: 1790919 (The Pile)

Max Activating Token Index: 616

Text #12

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.6625. Min Act: -0.0000

Data Index: 1088061 (The Pile)

Max Activating Token Index: 805

Text #13

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.6780. Min Act: -0.0000

Data Index: 1780088 (The Pile)

Max Activating Token Index: 876

Text #14

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.6401. Min Act: -0.0000

Data Index: 695371 (The Pile)

Max Activating Token Index: 301

Text #15

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.6169. Min Act: -0.0000

Data Index: 382515 (The Pile)

Max Activating Token Index: 219

Text #16

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.6189. Min Act: -0.0000

Data Index: 932168 (The Pile)

Max Activating Token Index: 364

Text #17

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.6519. Min Act: -0.0000

Data Index: 1706202 (The Pile)

Max Activating Token Index: 786

Text #18

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.6351. Min Act: -0.0000

Data Index: 861088 (The Pile)

Max Activating Token Index: 391

Text #19

Max Range: 0.8848. Min Range: -0.8848

Max Act: 0.6234. Min Act: -0.0000

Data Index: 679088 (The Pile)

Max Activating Token Index: 501