Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2618 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2240. Min Range: -1.2240

Max Act: 1.2240. Min Act: -0.0000

Data Index: 63945 (The Pile)

Max Activating Token Index: 893

Text #1

Max Range: 1.2240. Min Range: -1.2240

Max Act: 1.0327. Min Act: -0.0000

Data Index: 1283289 (The Pile)

Max Activating Token Index: 38

Text #2

Max Range: 1.2240. Min Range: -1.2240

Max Act: 1.0210. Min Act: -0.0000

Data Index: 215294 (The Pile)

Max Activating Token Index: 520

Text #3

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.9338. Min Act: -0.0000

Data Index: 1533015 (The Pile)

Max Activating Token Index: 318

Text #4

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.9722. Min Act: -0.0000

Data Index: 247293 (The Pile)

Max Activating Token Index: 565

Text #5

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.8947. Min Act: -0.0000

Data Index: 332604 (The Pile)

Max Activating Token Index: 496

Text #6

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.8918. Min Act: -0.0000

Data Index: 139756 (The Pile)

Max Activating Token Index: 221

Text #7

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.8313. Min Act: -0.0000

Data Index: 953794 (The Pile)

Max Activating Token Index: 203

Text #8

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.9137. Min Act: -0.0000

Data Index: 1557953 (The Pile)

Max Activating Token Index: 165

Text #9

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.8361. Min Act: -0.0000

Data Index: 395038 (The Pile)

Max Activating Token Index: 1010

Text #10

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.8438. Min Act: -0.0000

Data Index: 1582092 (The Pile)

Max Activating Token Index: 260

Text #11

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.8381. Min Act: -0.0000

Data Index: 1173122 (The Pile)

Max Activating Token Index: 265

Text #12

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.7879. Min Act: -0.0000

Data Index: 474278 (The Pile)

Max Activating Token Index: 751

Text #13

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.8470. Min Act: -0.0000

Data Index: 92509 (The Pile)

Max Activating Token Index: 260

Text #14

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.8268. Min Act: -0.0000

Data Index: 1901681 (The Pile)

Max Activating Token Index: 705

Text #15

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.7954. Min Act: -0.0000

Data Index: 1967912 (The Pile)

Max Activating Token Index: 469

Text #16

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.6943. Min Act: -0.0000

Data Index: 1865947 (The Pile)

Max Activating Token Index: 860

Text #17

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.8272. Min Act: -0.0000

Data Index: 903230 (The Pile)

Max Activating Token Index: 769

Text #18

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.7420. Min Act: -0.0000

Data Index: 562869 (The Pile)

Max Activating Token Index: 461

Text #19

Max Range: 1.2240. Min Range: -1.2240

Max Act: 0.7855. Min Act: -0.0000

Data Index: 1353767 (The Pile)

Max Activating Token Index: 264