Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3915 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.4524. Min Act: -0.0000

Data Index: 1988143 (The Pile)

Max Activating Token Index: 617

Text #1

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.4168. Min Act: -0.0000

Data Index: 1465149 (The Pile)

Max Activating Token Index: 602

Text #2

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.4019. Min Act: -0.0000

Data Index: 217982 (The Pile)

Max Activating Token Index: 701

Text #3

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.4284. Min Act: -0.0000

Data Index: 1185043 (The Pile)

Max Activating Token Index: 938

Text #4

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.3883. Min Act: -0.0000

Data Index: 1574177 (The Pile)

Max Activating Token Index: 698

Text #5

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.3859. Min Act: -0.0000

Data Index: 559793 (The Pile)

Max Activating Token Index: 289

Text #6

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.3845. Min Act: -0.0000

Data Index: 470073 (The Pile)

Max Activating Token Index: 266

Text #7

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.3808. Min Act: -0.0000

Data Index: 918271 (The Pile)

Max Activating Token Index: 186

Text #8

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.3544. Min Act: -0.0000

Data Index: 709893 (The Pile)

Max Activating Token Index: 264

Text #9

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.3382. Min Act: -0.0000

Data Index: 326889 (The Pile)

Max Activating Token Index: 804

Text #10

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.3792. Min Act: -0.0000

Data Index: 1644143 (The Pile)

Max Activating Token Index: 748

Text #11

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.3156. Min Act: -0.0000

Data Index: 1810373 (The Pile)

Max Activating Token Index: 510

Text #12

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.3322. Min Act: -0.0000

Data Index: 157242 (The Pile)

Max Activating Token Index: 89

Text #13

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.3210. Min Act: -0.0000

Data Index: 762494 (The Pile)

Max Activating Token Index: 222

Text #14

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.3320. Min Act: -0.0000

Data Index: 1596747 (The Pile)

Max Activating Token Index: 78

Text #15

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.2845. Min Act: -0.0000

Data Index: 586879 (The Pile)

Max Activating Token Index: 487

Text #16

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.2632. Min Act: -0.0000

Data Index: 294756 (The Pile)

Max Activating Token Index: 296

Text #17

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.2956. Min Act: -0.0000

Data Index: 26816 (The Pile)

Max Activating Token Index: 838

Text #18

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.2941. Min Act: -0.0000

Data Index: 560690 (The Pile)

Max Activating Token Index: 428

Text #19

Max Range: 0.4524. Min Range: -0.4524

Max Act: 0.2918. Min Act: -0.0000

Data Index: 322281 (The Pile)

Max Activating Token Index: 741