Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2938 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.3736. Min Act: -0.0000

Data Index: 1810060 (The Pile)

Max Activating Token Index: 828

Text #1

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.2889. Min Act: -0.0000

Data Index: 882119 (The Pile)

Max Activating Token Index: 976

Text #2

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.3329. Min Act: -0.0000

Data Index: 1638686 (The Pile)

Max Activating Token Index: 950

Text #3

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.3146. Min Act: -0.0000

Data Index: 1951954 (The Pile)

Max Activating Token Index: 207

Text #4

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.3229. Min Act: -0.0000

Data Index: 207656 (The Pile)

Max Activating Token Index: 610

Text #5

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.2960. Min Act: -0.0000

Data Index: 1878141 (The Pile)

Max Activating Token Index: 372

Text #6

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.3088. Min Act: -0.0000

Data Index: 1595319 (The Pile)

Max Activating Token Index: 226

Text #7

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.2747. Min Act: -0.0000

Data Index: 1869899 (The Pile)

Max Activating Token Index: 185

Text #8

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.2802. Min Act: -0.0000

Data Index: 951731 (The Pile)

Max Activating Token Index: 235

Text #9

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.2842. Min Act: -0.0000

Data Index: 738088 (The Pile)

Max Activating Token Index: 503

Text #10

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.2635. Min Act: -0.0000

Data Index: 406304 (The Pile)

Max Activating Token Index: 167

Text #11

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.2574. Min Act: -0.0000

Data Index: 1614048 (The Pile)

Max Activating Token Index: 44

Text #12

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.2648. Min Act: -0.0000

Data Index: 1474959 (The Pile)

Max Activating Token Index: 255

Text #13

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.2606. Min Act: -0.0000

Data Index: 1625011 (The Pile)

Max Activating Token Index: 139

Text #14

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.2387. Min Act: -0.0000

Data Index: 824745 (The Pile)

Max Activating Token Index: 790

Text #15

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.2146. Min Act: -0.0000

Data Index: 1216643 (The Pile)

Max Activating Token Index: 108

Text #16

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.2176. Min Act: -0.0000

Data Index: 1786596 (The Pile)

Max Activating Token Index: 865

Text #17

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.2504. Min Act: -0.0000

Data Index: 936576 (The Pile)

Max Activating Token Index: 439

Text #18

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.2450. Min Act: -0.0000

Data Index: 1346542 (The Pile)

Max Activating Token Index: 373

Text #19

Max Range: 0.3736. Min Range: -0.3736

Max Act: 0.2352. Min Act: -0.0000

Data Index: 1094047 (The Pile)

Max Activating Token Index: 160