Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2281 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1962. Min Act: -0.0000

Data Index: 211854 (The Pile)

Max Activating Token Index: 704

Text #1

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1875. Min Act: -0.0000

Data Index: 908566 (The Pile)

Max Activating Token Index: 627

Text #2

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1591. Min Act: -0.0000

Data Index: 93050 (The Pile)

Max Activating Token Index: 998

Text #3

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1501. Min Act: -0.0000

Data Index: 741538 (The Pile)

Max Activating Token Index: 162

Text #4

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1365. Min Act: -0.0000

Data Index: 1448630 (The Pile)

Max Activating Token Index: 558

Text #5

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1483. Min Act: -0.0000

Data Index: 136148 (The Pile)

Max Activating Token Index: 480

Text #6

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1369. Min Act: -0.0000

Data Index: 1464406 (The Pile)

Max Activating Token Index: 880

Text #7

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1411. Min Act: -0.0000

Data Index: 673671 (The Pile)

Max Activating Token Index: 34

Text #8

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1385. Min Act: -0.0000

Data Index: 188230 (The Pile)

Max Activating Token Index: 17

Text #9

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1348. Min Act: -0.0000

Data Index: 436925 (The Pile)

Max Activating Token Index: 927

Text #10

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1270. Min Act: -0.0000

Data Index: 1462089 (The Pile)

Max Activating Token Index: 253

Text #11

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1390. Min Act: -0.0000

Data Index: 897708 (The Pile)

Max Activating Token Index: 210

Text #12

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1132. Min Act: -0.0000

Data Index: 378907 (The Pile)

Max Activating Token Index: 982

Text #13

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1246. Min Act: -0.0000

Data Index: 1791319 (The Pile)

Max Activating Token Index: 456

Text #14

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1377. Min Act: -0.0000

Data Index: 1407033 (The Pile)

Max Activating Token Index: 710

Text #15

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1302. Min Act: -0.0000

Data Index: 1846156 (The Pile)

Max Activating Token Index: 16

Text #16

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1309. Min Act: -0.0000

Data Index: 1331811 (The Pile)

Max Activating Token Index: 581

Text #17

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1265. Min Act: -0.0000

Data Index: 208385 (The Pile)

Max Activating Token Index: 784

Text #18

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1280. Min Act: -0.0000

Data Index: 462124 (The Pile)

Max Activating Token Index: 97

Text #19

Max Range: 0.1962. Min Range: -0.1962

Max Act: 0.1245. Min Act: -0.0000

Data Index: 1995116 (The Pile)

Max Activating Token Index: 52