Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1102 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.5156. Min Act: -0.0000

Data Index: 1113882 (The Pile)

Max Activating Token Index: 304

Text #1

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.3008. Min Act: -0.0000

Data Index: 1632825 (The Pile)

Max Activating Token Index: 680

Text #2

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.4899. Min Act: -0.0000

Data Index: 47151 (The Pile)

Max Activating Token Index: 372

Text #3

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.1828. Min Act: -0.0000

Data Index: 920052 (The Pile)

Max Activating Token Index: 488

Text #4

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.2497. Min Act: -0.0000

Data Index: 607054 (The Pile)

Max Activating Token Index: 682

Text #5

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.2261. Min Act: -0.0000

Data Index: 907141 (The Pile)

Max Activating Token Index: 553

Text #6

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.3327. Min Act: -0.0000

Data Index: 1670881 (The Pile)

Max Activating Token Index: 635

Text #7

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.1983. Min Act: -0.0000

Data Index: 1361296 (The Pile)

Max Activating Token Index: 762

Text #8

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.2433. Min Act: -0.0000

Data Index: 307713 (The Pile)

Max Activating Token Index: 479

Text #9

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.1348. Min Act: -0.0000

Data Index: 1397046 (The Pile)

Max Activating Token Index: 612

Text #10

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.1748. Min Act: -0.0000

Data Index: 707710 (The Pile)

Max Activating Token Index: 477

Text #11

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.3380. Min Act: -0.0000

Data Index: 982158 (The Pile)

Max Activating Token Index: 361

Text #12

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.1812. Min Act: -0.0000

Data Index: 908020 (The Pile)

Max Activating Token Index: 929

Text #13

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.2519. Min Act: -0.0000

Data Index: 241646 (The Pile)

Max Activating Token Index: 650

Text #14

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.1527. Min Act: -0.0000

Data Index: 42805 (The Pile)

Max Activating Token Index: 438

Text #15

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.1456. Min Act: 0.0000

Data Index: 698417 (The Pile)

Max Activating Token Index: 140

Text #16

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.0655. Min Act: -0.0000

Data Index: 1910963 (The Pile)

Max Activating Token Index: 794

Text #17

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.0068. Min Act: -0.0000

Data Index: 1262385 (The Pile)

Max Activating Token Index: 921

Text #18

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.2302. Min Act: -0.0000

Data Index: 153697 (The Pile)

Max Activating Token Index: 772

Text #19

Max Range: 1.5156. Min Range: -1.5156

Max Act: 1.0740. Min Act: -0.0000

Data Index: 1526103 (The Pile)

Max Activating Token Index: 511