Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4281 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.6120. Min Act: -0.0000

Data Index: 109338 (The Pile)

Max Activating Token Index: 1018

Text #1

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.6933. Min Act: -0.0000

Data Index: 987226 (The Pile)

Max Activating Token Index: 163

Text #2

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.5625. Min Act: -0.0000

Data Index: 187985 (The Pile)

Max Activating Token Index: 460

Text #3

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.4453. Min Act: -0.0000

Data Index: 1431082 (The Pile)

Max Activating Token Index: 1015

Text #4

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.5263. Min Act: -0.0000

Data Index: 883054 (The Pile)

Max Activating Token Index: 182

Text #5

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.5473. Min Act: -0.0000

Data Index: 99791 (The Pile)

Max Activating Token Index: 169

Text #6

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.5267. Min Act: -0.0000

Data Index: 1176154 (The Pile)

Max Activating Token Index: 264

Text #7

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.5017. Min Act: -0.0000

Data Index: 1660892 (The Pile)

Max Activating Token Index: 237

Text #8

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.4847. Min Act: -0.0000

Data Index: 1863759 (The Pile)

Max Activating Token Index: 287

Text #9

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.4958. Min Act: -0.0000

Data Index: 1531146 (The Pile)

Max Activating Token Index: 42

Text #10

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.4982. Min Act: -0.0000

Data Index: 1750114 (The Pile)

Max Activating Token Index: 726

Text #11

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.4657. Min Act: -0.0000

Data Index: 1863942 (The Pile)

Max Activating Token Index: 471

Text #12

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.4346. Min Act: -0.0000

Data Index: 1179092 (The Pile)

Max Activating Token Index: 243

Text #13

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.4694. Min Act: -0.0000

Data Index: 1115072 (The Pile)

Max Activating Token Index: 102

Text #14

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.4705. Min Act: -0.0000

Data Index: 617436 (The Pile)

Max Activating Token Index: 219

Text #15

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.4473. Min Act: -0.0000

Data Index: 1806816 (The Pile)

Max Activating Token Index: 400

Text #16

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.4345. Min Act: -0.0000

Data Index: 819300 (The Pile)

Max Activating Token Index: 122

Text #17

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.5284. Min Act: -0.0000

Data Index: 699653 (The Pile)

Max Activating Token Index: 998

Text #18

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.4372. Min Act: -0.0000

Data Index: 226300 (The Pile)

Max Activating Token Index: 67

Text #19

Max Range: 0.6933. Min Range: -0.6933

Max Act: 0.4531. Min Act: -0.0000

Data Index: 497558 (The Pile)

Max Activating Token Index: 539