Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4148 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 7.1083. Min Range: -7.1083

Max Act: 7.0368. Min Act: -0.0000

Data Index: 1832010 (The Pile)

Max Activating Token Index: 332

Text #1

Max Range: 7.1083. Min Range: -7.1083

Max Act: 6.7879. Min Act: -0.0000

Data Index: 1607910 (The Pile)

Max Activating Token Index: 241

Text #2

Max Range: 7.1083. Min Range: -7.1083

Max Act: 6.7707. Min Act: -0.0000

Data Index: 1284776 (The Pile)

Max Activating Token Index: 385

Text #3

Max Range: 7.1083. Min Range: -7.1083

Max Act: 6.6819. Min Act: -0.0000

Data Index: 1420600 (The Pile)

Max Activating Token Index: 289

Text #4

Max Range: 7.1083. Min Range: -7.1083

Max Act: 6.8459. Min Act: -0.0000

Data Index: 5715 (The Pile)

Max Activating Token Index: 351

Text #5

Max Range: 7.1083. Min Range: -7.1083

Max Act: 6.8451. Min Act: -0.0000

Data Index: 1293021 (The Pile)

Max Activating Token Index: 404

Text #6

Max Range: 7.1083. Min Range: -7.1083

Max Act: 6.4305. Min Act: -0.0000

Data Index: 1483335 (The Pile)

Max Activating Token Index: 249

Text #7

Max Range: 7.1083. Min Range: -7.1083

Max Act: 7.1083. Min Act: -0.0000

Data Index: 938786 (The Pile)

Max Activating Token Index: 835

Text #8

Max Range: 7.1083. Min Range: -7.1083

Max Act: 6.2292. Min Act: -0.0000

Data Index: 1711396 (The Pile)

Max Activating Token Index: 629

Text #9

Max Range: 7.1083. Min Range: -7.1083

Max Act: 6.5770. Min Act: -0.0000

Data Index: 793928 (The Pile)

Max Activating Token Index: 677

Text #10

Max Range: 7.1083. Min Range: -7.1083

Max Act: 6.8675. Min Act: -0.0000

Data Index: 1567935 (The Pile)

Max Activating Token Index: 494

Text #11

Max Range: 7.1083. Min Range: -7.1083

Max Act: 6.6336. Min Act: -0.0000

Data Index: 1881207 (The Pile)

Max Activating Token Index: 383

Text #12

Max Range: 7.1083. Min Range: -7.1083

Max Act: 6.3597. Min Act: -0.0000

Data Index: 118644 (The Pile)

Max Activating Token Index: 182

Text #13

Max Range: 7.1083. Min Range: -7.1083

Max Act: 6.1241. Min Act: -0.0000

Data Index: 762748 (The Pile)

Max Activating Token Index: 641

Text #14

Max Range: 7.1083. Min Range: -7.1083

Max Act: 6.2471. Min Act: -0.0000

Data Index: 512797 (The Pile)

Max Activating Token Index: 470

Text #15

Max Range: 7.1083. Min Range: -7.1083

Max Act: 5.9441. Min Act: -0.0000

Data Index: 455584 (The Pile)

Max Activating Token Index: 433

Text #16

Max Range: 7.1083. Min Range: -7.1083

Max Act: 6.0861. Min Act: -0.0000

Data Index: 85524 (The Pile)

Max Activating Token Index: 362

Text #17

Max Range: 7.1083. Min Range: -7.1083

Max Act: 5.9285. Min Act: -0.0000

Data Index: 1262663 (The Pile)

Max Activating Token Index: 518

Text #18

Max Range: 7.1083. Min Range: -7.1083

Max Act: 6.1708. Min Act: -0.0000

Data Index: 919337 (The Pile)

Max Activating Token Index: 256

Text #19

Max Range: 7.1083. Min Range: -7.1083

Max Act: 5.8409. Min Act: -0.0000

Data Index: 504345 (The Pile)

Max Activating Token Index: 398