Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 617 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.2398. Min Act: -0.0000

Data Index: 100035 (The Pile)

Max Activating Token Index: 422

Text #1

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.2197. Min Act: -0.0000

Data Index: 396041 (The Pile)

Max Activating Token Index: 754

Text #2

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.2102. Min Act: -0.0000

Data Index: 791797 (The Pile)

Max Activating Token Index: 418

Text #3

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1735. Min Act: -0.0000

Data Index: 1524199 (The Pile)

Max Activating Token Index: 878

Text #4

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1500. Min Act: -0.0000

Data Index: 457306 (The Pile)

Max Activating Token Index: 308

Text #5

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1590. Min Act: -0.0000

Data Index: 147850 (The Pile)

Max Activating Token Index: 523

Text #6

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1482. Min Act: -0.0000

Data Index: 1393212 (The Pile)

Max Activating Token Index: 825

Text #7

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1614. Min Act: -0.0000

Data Index: 427914 (The Pile)

Max Activating Token Index: 535

Text #8

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1429. Min Act: -0.0000

Data Index: 1386197 (The Pile)

Max Activating Token Index: 788

Text #9

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1387. Min Act: -0.0000

Data Index: 917063 (The Pile)

Max Activating Token Index: 261

Text #10

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1462. Min Act: -0.0000

Data Index: 730771 (The Pile)

Max Activating Token Index: 561

Text #11

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1329. Min Act: -0.0000

Data Index: 696286 (The Pile)

Max Activating Token Index: 284

Text #12

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1309. Min Act: -0.0000

Data Index: 42786 (The Pile)

Max Activating Token Index: 582

Text #13

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1252. Min Act: -0.0000

Data Index: 1641109 (The Pile)

Max Activating Token Index: 80

Text #14

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1176. Min Act: -0.0000

Data Index: 788843 (The Pile)

Max Activating Token Index: 961

Text #15

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1143. Min Act: -0.0000

Data Index: 726580 (The Pile)

Max Activating Token Index: 257

Text #16

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1157. Min Act: -0.0000

Data Index: 1039128 (The Pile)

Max Activating Token Index: 344

Text #17

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1480. Min Act: -0.0000

Data Index: 1580120 (The Pile)

Max Activating Token Index: 1008

Text #18

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1192. Min Act: -0.0000

Data Index: 1460746 (The Pile)

Max Activating Token Index: 676

Text #19

Max Range: 0.2398. Min Range: -0.2398

Max Act: 0.1187. Min Act: -0.0000

Data Index: 1188136 (The Pile)

Max Activating Token Index: 244