Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 4599 in Layer 11

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 5.1482. Min Range: -5.1482

Max Act: 5.1482. Min Act: -0.0000

Data Index: 1525063 (Python Code)

Max Activating Token Index: 768

Text #1

Max Range: 5.1482. Min Range: -5.1482

Max Act: 4.4886. Min Act: -0.0000

Data Index: 1601055 (Python Code)

Max Activating Token Index: 269

Text #2

Max Range: 5.1482. Min Range: -5.1482

Max Act: 3.7424. Min Act: -0.0000

Data Index: 1527842 (Python Code)

Max Activating Token Index: 862

Text #3

Max Range: 5.1482. Min Range: -5.1482

Max Act: 3.8620. Min Act: -0.0000

Data Index: 1385043 (Python Code)

Max Activating Token Index: 125

Text #4

Max Range: 5.1482. Min Range: -5.1482

Max Act: 3.4951. Min Act: -0.0000

Data Index: 1559915 (Python Code)

Max Activating Token Index: 782

Text #5

Max Range: 5.1482. Min Range: -5.1482

Max Act: 3.2499. Min Act: -0.0000

Data Index: 1396202 (Python Code)

Max Activating Token Index: 821

Text #6

Max Range: 5.1482. Min Range: -5.1482

Max Act: 2.7468. Min Act: -0.0000

Data Index: 1412922 (Python Code)

Max Activating Token Index: 444

Text #7

Max Range: 5.1482. Min Range: -5.1482

Max Act: 3.1372. Min Act: -0.0000

Data Index: 1549736 (Python Code)

Max Activating Token Index: 967

Text #8

Max Range: 5.1482. Min Range: -5.1482

Max Act: 2.5683. Min Act: -0.0000

Data Index: 1578624 (Python Code)

Max Activating Token Index: 337

Text #9

Max Range: 5.1482. Min Range: -5.1482

Max Act: 2.3791. Min Act: -0.0000

Data Index: 1223756 (C4 (Web Text))

Max Activating Token Index: 926

Text #10

Max Range: 5.1482. Min Range: -5.1482

Max Act: 2.3668. Min Act: -0.0000

Data Index: 1639302 (Python Code)

Max Activating Token Index: 830

Text #11

Max Range: 5.1482. Min Range: -5.1482

Max Act: 1.9896. Min Act: -0.0000

Data Index: 1477382 (Python Code)

Max Activating Token Index: 365

Text #12

Max Range: 5.1482. Min Range: -5.1482

Max Act: 2.2982. Min Act: -0.0000

Data Index: 1403418 (Python Code)

Max Activating Token Index: 218

Text #13

Max Range: 5.1482. Min Range: -5.1482

Max Act: 2.1996. Min Act: -0.0000

Data Index: 368607 (C4 (Web Text))

Max Activating Token Index: 529

Text #14

Max Range: 5.1482. Min Range: -5.1482

Max Act: 2.1427. Min Act: -0.0000

Data Index: 1360032 (Python Code)

Max Activating Token Index: 450

Text #15

Max Range: 5.1482. Min Range: -5.1482

Max Act: 2.2002. Min Act: -0.0000

Data Index: 228438 (C4 (Web Text))

Max Activating Token Index: 647

Text #16

Max Range: 5.1482. Min Range: -5.1482

Max Act: 2.0735. Min Act: -0.0000

Data Index: 300493 (C4 (Web Text))

Max Activating Token Index: 276

Text #17

Max Range: 5.1482. Min Range: -5.1482

Max Act: 1.8859. Min Act: -0.0000

Data Index: 1416752 (Python Code)

Max Activating Token Index: 688

Text #18

Max Range: 5.1482. Min Range: -5.1482

Max Act: 2.0751. Min Act: -0.0000

Data Index: 81826 (C4 (Web Text))

Max Activating Token Index: 660

Text #19

Max Range: 5.1482. Min Range: -5.1482

Max Act: 1.9693. Min Act: -0.0000

Data Index: 1468741 (Python Code)

Max Activating Token Index: 420