Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3641 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.8552. Min Act: -0.0000

Data Index: 518453 (The Pile)

Max Activating Token Index: 245

Text #1

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.9639. Min Act: -0.0000

Data Index: 1569750 (The Pile)

Max Activating Token Index: 238

Text #2

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.8706. Min Act: -0.0000

Data Index: 1207716 (The Pile)

Max Activating Token Index: 596

Text #3

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.8481. Min Act: -0.0000

Data Index: 288687 (The Pile)

Max Activating Token Index: 160

Text #4

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.8009. Min Act: -0.0000

Data Index: 765342 (The Pile)

Max Activating Token Index: 500

Text #5

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.6687. Min Act: -0.0000

Data Index: 798470 (The Pile)

Max Activating Token Index: 76

Text #6

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.8127. Min Act: -0.0000

Data Index: 9920 (The Pile)

Max Activating Token Index: 75

Text #7

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.6650. Min Act: -0.0000

Data Index: 1736246 (The Pile)

Max Activating Token Index: 164

Text #8

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.6943. Min Act: -0.0000

Data Index: 265287 (The Pile)

Max Activating Token Index: 313

Text #9

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.6932. Min Act: -0.0000

Data Index: 1526481 (The Pile)

Max Activating Token Index: 848

Text #10

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.6677. Min Act: -0.0000

Data Index: 1334363 (The Pile)

Max Activating Token Index: 917

Text #11

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.7222. Min Act: -0.0000

Data Index: 1379774 (The Pile)

Max Activating Token Index: 292

Text #12

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.6787. Min Act: -0.0000

Data Index: 869379 (The Pile)

Max Activating Token Index: 265

Text #13

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.7021. Min Act: -0.0000

Data Index: 1615043 (The Pile)

Max Activating Token Index: 886

Text #14

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.6952. Min Act: -0.0000

Data Index: 1094328 (The Pile)

Max Activating Token Index: 61

Text #15

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.6702. Min Act: -0.0000

Data Index: 1541930 (The Pile)

Max Activating Token Index: 512

Text #16

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.6225. Min Act: -0.0000

Data Index: 757955 (The Pile)

Max Activating Token Index: 936

Text #17

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.6511. Min Act: -0.0000

Data Index: 1405478 (The Pile)

Max Activating Token Index: 359

Text #18

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.6672. Min Act: -0.0000

Data Index: 262386 (The Pile)

Max Activating Token Index: 307

Text #19

Max Range: 0.9639. Min Range: -0.9639

Max Act: 0.6271. Min Act: -0.0000

Data Index: 1432928 (The Pile)

Max Activating Token Index: 956