Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2817 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('solu-12l')

Click the toggle to see the full text!

Text #0

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.1729. Min Act: -0.0002

Data Index: 1504873 (Python Code)

Max Activating Token Index: 667

Text #1

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.1491. Min Act: -0.0002

Data Index: 1582007 (Python Code)

Max Activating Token Index: 229

Text #2

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.1306. Min Act: -0.0002

Data Index: 1643987 (Python Code)

Max Activating Token Index: 681

Text #3

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.1202. Min Act: -0.0002

Data Index: 1479816 (Python Code)

Max Activating Token Index: 877

Text #4

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.0968. Min Act: -0.0002

Data Index: 1366445 (Python Code)

Max Activating Token Index: 770

Text #5

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.1118. Min Act: -0.0002

Data Index: 1561179 (Python Code)

Max Activating Token Index: 84

Text #6

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.0711. Min Act: -0.0002

Data Index: 1485267 (Python Code)

Max Activating Token Index: 936

Text #7

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.0829. Min Act: -0.0002

Data Index: 1583723 (Python Code)

Max Activating Token Index: 937

Text #8

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.0916. Min Act: -0.0002

Data Index: 1520389 (Python Code)

Max Activating Token Index: 994

Text #9

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.0614. Min Act: -0.0002

Data Index: 1578704 (Python Code)

Max Activating Token Index: 281

Text #10

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.0762. Min Act: -0.0002

Data Index: 1368318 (Python Code)

Max Activating Token Index: 652

Text #11

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.0698. Min Act: -0.0002

Data Index: 1506940 (Python Code)

Max Activating Token Index: 395

Text #12

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.0597. Min Act: -0.0002

Data Index: 1382427 (Python Code)

Max Activating Token Index: 120

Text #13

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.0435. Min Act: -0.0002

Data Index: 1605557 (Python Code)

Max Activating Token Index: 364

Text #14

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.0351. Min Act: -0.0002

Data Index: 1625712 (Python Code)

Max Activating Token Index: 944

Text #15

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.0416. Min Act: -0.0002

Data Index: 1538856 (Python Code)

Max Activating Token Index: 309

Text #16

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.0565. Min Act: -0.0002

Data Index: 1637607 (Python Code)

Max Activating Token Index: 933

Text #17

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.0672. Min Act: -0.0002

Data Index: 1365439 (Python Code)

Max Activating Token Index: 754

Text #18

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.0499. Min Act: -0.0002

Data Index: 1446935 (Python Code)

Max Activating Token Index: 415

Text #19

Max Range: 1.1729. Min Range: -1.1729

Max Act: 1.0580. Min Act: -0.0002

Data Index: 1648546 (Python Code)

Max Activating Token Index: 364