Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1094 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l')

Text #0

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6858. Min Act: -0.0000

Data Index: 1517479 (Python Code)

Max Activating Token Index: 921

Text #1

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.5850. Min Act: -0.0000

Data Index: 1592159 (Python Code)

Max Activating Token Index: 828

Text #2

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.5684. Min Act: -0.0000

Data Index: 1439576 (Python Code)

Max Activating Token Index: 855

Text #3

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6590. Min Act: -0.0000

Data Index: 1383621 (Python Code)

Max Activating Token Index: 884

Text #4

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6550. Min Act: -0.0000

Data Index: 1389246 (Python Code)

Max Activating Token Index: 666

Text #5

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.5580. Min Act: -0.0000

Data Index: 1564776 (Python Code)

Max Activating Token Index: 239

Text #6

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.5082. Min Act: -0.0000

Data Index: 1642033 (Python Code)

Max Activating Token Index: 1014

Text #7

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.5398. Min Act: -0.0000

Data Index: 1599727 (Python Code)

Max Activating Token Index: 63

Text #8

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.5823. Min Act: -0.0000

Data Index: 1521713 (Python Code)

Max Activating Token Index: 1009

Text #9

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.4133. Min Act: -0.0000

Data Index: 1636184 (Python Code)

Max Activating Token Index: 706

Text #10

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.5325. Min Act: -0.0000

Data Index: 1597186 (Python Code)

Max Activating Token Index: 776

Text #11

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.4398. Min Act: -0.0000

Data Index: 1594683 (Python Code)

Max Activating Token Index: 184

Text #12

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6567. Min Act: -0.0000

Data Index: 1457833 (Python Code)

Max Activating Token Index: 747

Text #13

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.5678. Min Act: -0.0000

Data Index: 1531005 (Python Code)

Max Activating Token Index: 166

Text #14

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.4720. Min Act: -0.0001

Data Index: 1616993 (Python Code)

Max Activating Token Index: 963

Text #15

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.4602. Min Act: -0.0000

Data Index: 1490752 (Python Code)

Max Activating Token Index: 49

Text #16

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.5324. Min Act: -0.0000

Data Index: 1522445 (Python Code)

Max Activating Token Index: 636

Text #17

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.4816. Min Act: -0.0000

Data Index: 1604477 (Python Code)

Max Activating Token Index: 781

Text #18

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.3989. Min Act: -0.0000

Data Index: 1618183 (Python Code)

Max Activating Token Index: 645

Text #19

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.4422. Min Act: -0.0000

Data Index: 1581992 (Python Code)

Max Activating Token Index: 1023