Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4803 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.7626. Min Act: -0.0000

Data Index: 391497 (The Pile)

Max Activating Token Index: 510

Text #1

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.5087. Min Act: -0.0000

Data Index: 511620 (The Pile)

Max Activating Token Index: 123

Text #2

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.5115. Min Act: -0.0000

Data Index: 169487 (The Pile)

Max Activating Token Index: 649

Text #3

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.6654. Min Act: -0.0000

Data Index: 1897792 (The Pile)

Max Activating Token Index: 437

Text #4

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.4646. Min Act: -0.0000

Data Index: 33202 (The Pile)

Max Activating Token Index: 184

Text #5

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.7832. Min Act: -0.0000

Data Index: 435064 (The Pile)

Max Activating Token Index: 472

Text #6

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.5618. Min Act: -0.0000

Data Index: 1624520 (The Pile)

Max Activating Token Index: 96

Text #7

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.3791. Min Act: -0.0000

Data Index: 1808841 (The Pile)

Max Activating Token Index: 303

Text #8

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.5142. Min Act: -0.0000

Data Index: 1021245 (The Pile)

Max Activating Token Index: 538

Text #9

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.4941. Min Act: -0.0000

Data Index: 1944564 (The Pile)

Max Activating Token Index: 423

Text #10

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.2959. Min Act: -0.0000

Data Index: 610730 (The Pile)

Max Activating Token Index: 885

Text #11

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.6407. Min Act: -0.0000

Data Index: 1198092 (The Pile)

Max Activating Token Index: 885

Text #12

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.2533. Min Act: -0.0000

Data Index: 1386968 (The Pile)

Max Activating Token Index: 941

Text #13

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.2341. Min Act: -0.0000

Data Index: 108604 (The Pile)

Max Activating Token Index: 59

Text #14

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.5836. Min Act: -0.0000

Data Index: 36416 (The Pile)

Max Activating Token Index: 53

Text #15

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.3501. Min Act: -0.0000

Data Index: 1342799 (The Pile)

Max Activating Token Index: 864

Text #16

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.5281. Min Act: -0.0000

Data Index: 909460 (The Pile)

Max Activating Token Index: 456

Text #17

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.3314. Min Act: -0.0000

Data Index: 1320442 (The Pile)

Max Activating Token Index: 422

Text #18

Max Range: 5.7832. Min Range: -5.7832

Max Act: 5.3286. Min Act: -0.0000

Data Index: 714283 (The Pile)

Max Activating Token Index: 431

Text #19

Max Range: 5.7832. Min Range: -5.7832

Max Act: 4.9547. Min Act: -0.0000

Data Index: 912211 (The Pile)

Max Activating Token Index: 744