Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3360 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1627. Min Act: -0.0000

Data Index: 346191 (The Pile)

Max Activating Token Index: 326

Text #1

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1532. Min Act: -0.0000

Data Index: 1060393 (The Pile)

Max Activating Token Index: 160

Text #2

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1522. Min Act: -0.0000

Data Index: 964135 (The Pile)

Max Activating Token Index: 509

Text #3

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1436. Min Act: -0.0000

Data Index: 1071584 (The Pile)

Max Activating Token Index: 216

Text #4

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1424. Min Act: -0.0000

Data Index: 183232 (The Pile)

Max Activating Token Index: 650

Text #5

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1376. Min Act: -0.0000

Data Index: 1075126 (The Pile)

Max Activating Token Index: 629

Text #6

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1517. Min Act: -0.0000

Data Index: 1332882 (The Pile)

Max Activating Token Index: 666

Text #7

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1307. Min Act: -0.0000

Data Index: 154379 (The Pile)

Max Activating Token Index: 625

Text #8

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1285. Min Act: -0.0000

Data Index: 1865651 (The Pile)

Max Activating Token Index: 1013

Text #9

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1181. Min Act: -0.0000

Data Index: 1835988 (The Pile)

Max Activating Token Index: 617

Text #10

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1134. Min Act: -0.0000

Data Index: 1064360 (The Pile)

Max Activating Token Index: 632

Text #11

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1339. Min Act: -0.0000

Data Index: 902994 (The Pile)

Max Activating Token Index: 644

Text #12

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1377. Min Act: -0.0000

Data Index: 1445628 (The Pile)

Max Activating Token Index: 377

Text #13

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1381. Min Act: -0.0000

Data Index: 1621981 (The Pile)

Max Activating Token Index: 200

Text #14

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1344. Min Act: -0.0000

Data Index: 1805731 (The Pile)

Max Activating Token Index: 263

Text #15

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1299. Min Act: -0.0000

Data Index: 1657427 (The Pile)

Max Activating Token Index: 604

Text #16

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1257. Min Act: -0.0000

Data Index: 1711364 (The Pile)

Max Activating Token Index: 495

Text #17

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1264. Min Act: -0.0000

Data Index: 173147 (The Pile)

Max Activating Token Index: 703

Text #18

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1243. Min Act: -0.0000

Data Index: 1849739 (The Pile)

Max Activating Token Index: 994

Text #19

Max Range: 0.1627. Min Range: -0.1627

Max Act: 0.1249. Min Act: -0.0000

Data Index: 276451 (The Pile)

Max Activating Token Index: 809