Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 288 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2264. Min Range: -1.2264

Max Act: 1.2077. Min Act: -0.0000

Data Index: 1940097 (The Pile)

Max Activating Token Index: 220

Text #1

Max Range: 1.2264. Min Range: -1.2264

Max Act: 1.1436. Min Act: -0.0000

Data Index: 1149407 (The Pile)

Max Activating Token Index: 317

Text #2

Max Range: 1.2264. Min Range: -1.2264

Max Act: 1.2264. Min Act: -0.0000

Data Index: 453874 (The Pile)

Max Activating Token Index: 141

Text #3

Max Range: 1.2264. Min Range: -1.2264

Max Act: 0.8911. Min Act: -0.0000

Data Index: 818086 (The Pile)

Max Activating Token Index: 736

Text #4

Max Range: 1.2264. Min Range: -1.2264

Max Act: 1.0521. Min Act: -0.0000

Data Index: 1357184 (The Pile)

Max Activating Token Index: 963

Text #5

Max Range: 1.2264. Min Range: -1.2264

Max Act: 1.0238. Min Act: -0.0000

Data Index: 1370365 (The Pile)

Max Activating Token Index: 339

Text #6

Max Range: 1.2264. Min Range: -1.2264

Max Act: 0.9463. Min Act: -0.0000

Data Index: 1934068 (The Pile)

Max Activating Token Index: 296

Text #7

Max Range: 1.2264. Min Range: -1.2264

Max Act: 0.9375. Min Act: -0.0000

Data Index: 457317 (The Pile)

Max Activating Token Index: 250

Text #8

Max Range: 1.2264. Min Range: -1.2264

Max Act: 0.8955. Min Act: -0.0000

Data Index: 669162 (The Pile)

Max Activating Token Index: 985

Text #9

Max Range: 1.2264. Min Range: -1.2264

Max Act: 0.8862. Min Act: -0.0000

Data Index: 1580279 (The Pile)

Max Activating Token Index: 372

Text #10

Max Range: 1.2264. Min Range: -1.2264

Max Act: 1.0500. Min Act: -0.0000

Data Index: 1516500 (The Pile)

Max Activating Token Index: 1007

Text #11

Max Range: 1.2264. Min Range: -1.2264

Max Act: 1.0297. Min Act: -0.0000

Data Index: 917082 (The Pile)

Max Activating Token Index: 817

Text #12

Max Range: 1.2264. Min Range: -1.2264

Max Act: 0.9613. Min Act: -0.0000

Data Index: 1645607 (The Pile)

Max Activating Token Index: 494

Text #13

Max Range: 1.2264. Min Range: -1.2264

Max Act: 0.8194. Min Act: -0.0000

Data Index: 1212163 (The Pile)

Max Activating Token Index: 428

Text #14

Max Range: 1.2264. Min Range: -1.2264

Max Act: 0.7825. Min Act: -0.0000

Data Index: 1658679 (The Pile)

Max Activating Token Index: 572

Text #15

Max Range: 1.2264. Min Range: -1.2264

Max Act: 0.8294. Min Act: -0.0000

Data Index: 1619619 (The Pile)

Max Activating Token Index: 253

Text #16

Max Range: 1.2264. Min Range: -1.2264

Max Act: 0.7588. Min Act: -0.0000

Data Index: 1483538 (The Pile)

Max Activating Token Index: 463

Text #17

Max Range: 1.2264. Min Range: -1.2264

Max Act: 0.7396. Min Act: -0.0000

Data Index: 198729 (The Pile)

Max Activating Token Index: 45

Text #18

Max Range: 1.2264. Min Range: -1.2264

Max Act: 0.7246. Min Act: -0.0000

Data Index: 931930 (The Pile)

Max Activating Token Index: 496

Text #19

Max Range: 1.2264. Min Range: -1.2264

Max Act: 0.6542. Min Act: -0.0000

Data Index: 1431763 (The Pile)

Max Activating Token Index: 161