Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2295 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.5144. Min Act: -0.0000

Data Index: 1672080 (The Pile)

Max Activating Token Index: 81

Text #1

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.6648. Min Act: -0.0000

Data Index: 1285800 (The Pile)

Max Activating Token Index: 168

Text #2

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.6344. Min Act: -0.0000

Data Index: 1176147 (The Pile)

Max Activating Token Index: 552

Text #3

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.5158. Min Act: -0.0000

Data Index: 5368 (The Pile)

Max Activating Token Index: 930

Text #4

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.5964. Min Act: -0.0000

Data Index: 1901111 (The Pile)

Max Activating Token Index: 1019

Text #5

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.0503. Min Act: -0.0000

Data Index: 1042847 (The Pile)

Max Activating Token Index: 635

Text #6

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.4192. Min Act: -0.0000

Data Index: 1456008 (The Pile)

Max Activating Token Index: 447

Text #7

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.2461. Min Act: -0.0000

Data Index: 1540897 (The Pile)

Max Activating Token Index: 24

Text #8

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.0770. Min Act: -0.0000

Data Index: 1846956 (The Pile)

Max Activating Token Index: 435

Text #9

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.1762. Min Act: -0.0000

Data Index: 1732751 (The Pile)

Max Activating Token Index: 856

Text #10

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.1740. Min Act: -0.0000

Data Index: 1226396 (The Pile)

Max Activating Token Index: 377

Text #11

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.4591. Min Act: -0.0000

Data Index: 884948 (The Pile)

Max Activating Token Index: 493

Text #12

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.1347. Min Act: -0.0000

Data Index: 1686637 (The Pile)

Max Activating Token Index: 243

Text #13

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.2200. Min Act: -0.0000

Data Index: 307530 (The Pile)

Max Activating Token Index: 167

Text #14

Max Range: 3.6648. Min Range: -3.6648

Max Act: 2.9227. Min Act: -0.0000

Data Index: 1125133 (The Pile)

Max Activating Token Index: 482

Text #15

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.0438. Min Act: -0.0000

Data Index: 836469 (The Pile)

Max Activating Token Index: 537

Text #16

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.0219. Min Act: -0.0000

Data Index: 1719029 (The Pile)

Max Activating Token Index: 259

Text #17

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.0170. Min Act: -0.0000

Data Index: 521689 (The Pile)

Max Activating Token Index: 817

Text #18

Max Range: 3.6648. Min Range: -3.6648

Max Act: 3.1132. Min Act: -0.0000

Data Index: 413676 (The Pile)

Max Activating Token Index: 278

Text #19

Max Range: 3.6648. Min Range: -3.6648

Max Act: 2.8343. Min Act: -0.0000

Data Index: 997070 (The Pile)

Max Activating Token Index: 863