Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4751 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.2884. Min Range: -3.2884

Max Act: 3.2884. Min Act: -0.0000

Data Index: 408430 (The Pile)

Max Activating Token Index: 528

Text #1

Max Range: 3.2884. Min Range: -3.2884

Max Act: 2.7813. Min Act: -0.0000

Data Index: 492499 (The Pile)

Max Activating Token Index: 463

Text #2

Max Range: 3.2884. Min Range: -3.2884

Max Act: 2.6761. Min Act: -0.0000

Data Index: 1441013 (The Pile)

Max Activating Token Index: 322

Text #3

Max Range: 3.2884. Min Range: -3.2884

Max Act: 2.6388. Min Act: -0.0000

Data Index: 1268649 (The Pile)

Max Activating Token Index: 410

Text #4

Max Range: 3.2884. Min Range: -3.2884

Max Act: 2.5293. Min Act: -0.0000

Data Index: 611464 (The Pile)

Max Activating Token Index: 195

Text #5

Max Range: 3.2884. Min Range: -3.2884

Max Act: 2.2900. Min Act: -0.0000

Data Index: 1154867 (The Pile)

Max Activating Token Index: 406

Text #6

Max Range: 3.2884. Min Range: -3.2884

Max Act: 2.3444. Min Act: -0.0000

Data Index: 924678 (The Pile)

Max Activating Token Index: 409

Text #7

Max Range: 3.2884. Min Range: -3.2884

Max Act: 2.2210. Min Act: -0.0000

Data Index: 605638 (The Pile)

Max Activating Token Index: 709

Text #8

Max Range: 3.2884. Min Range: -3.2884

Max Act: 2.0553. Min Act: -0.0000

Data Index: 964354 (The Pile)

Max Activating Token Index: 207

Text #9

Max Range: 3.2884. Min Range: -3.2884

Max Act: 2.3156. Min Act: -0.0000

Data Index: 1499409 (The Pile)

Max Activating Token Index: 413

Text #10

Max Range: 3.2884. Min Range: -3.2884

Max Act: 2.1453. Min Act: -0.0000

Data Index: 1323477 (The Pile)

Max Activating Token Index: 348

Text #11

Max Range: 3.2884. Min Range: -3.2884

Max Act: 2.1399. Min Act: -0.0000

Data Index: 33109 (The Pile)

Max Activating Token Index: 333

Text #12

Max Range: 3.2884. Min Range: -3.2884

Max Act: 2.1368. Min Act: -0.0000

Data Index: 1307785 (The Pile)

Max Activating Token Index: 504

Text #13

Max Range: 3.2884. Min Range: -3.2884

Max Act: 2.1579. Min Act: -0.0000

Data Index: 1565548 (The Pile)

Max Activating Token Index: 313

Text #14

Max Range: 3.2884. Min Range: -3.2884

Max Act: 1.9886. Min Act: -0.0000

Data Index: 1952953 (The Pile)

Max Activating Token Index: 146

Text #15

Max Range: 3.2884. Min Range: -3.2884

Max Act: 2.1112. Min Act: -0.0000

Data Index: 1648229 (The Pile)

Max Activating Token Index: 236

Text #16

Max Range: 3.2884. Min Range: -3.2884

Max Act: 2.0916. Min Act: -0.0000

Data Index: 1664398 (The Pile)

Max Activating Token Index: 392

Text #17

Max Range: 3.2884. Min Range: -3.2884

Max Act: 1.9443. Min Act: -0.0000

Data Index: 1316270 (The Pile)

Max Activating Token Index: 320

Text #18

Max Range: 3.2884. Min Range: -3.2884

Max Act: 2.0151. Min Act: -0.0000

Data Index: 524870 (The Pile)

Max Activating Token Index: 585

Text #19

Max Range: 3.2884. Min Range: -3.2884

Max Act: 1.7173. Min Act: -0.0000

Data Index: 1485797 (The Pile)

Max Activating Token Index: 235