Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 448 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.3170. Min Act: -0.0000

Data Index: 314553 (The Pile)

Max Activating Token Index: 317

Text #1

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.2452. Min Act: -0.0000

Data Index: 325595 (The Pile)

Max Activating Token Index: 543

Text #2

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.1660. Min Act: -0.0000

Data Index: 634322 (The Pile)

Max Activating Token Index: 612

Text #3

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.2334. Min Act: -0.0000

Data Index: 961252 (The Pile)

Max Activating Token Index: 441

Text #4

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.1844. Min Act: -0.0000

Data Index: 674088 (The Pile)

Max Activating Token Index: 60

Text #5

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.1766. Min Act: -0.0000

Data Index: 1733766 (The Pile)

Max Activating Token Index: 220

Text #6

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.2125. Min Act: -0.0000

Data Index: 704957 (The Pile)

Max Activating Token Index: 317

Text #7

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.2726. Min Act: -0.0000

Data Index: 340059 (The Pile)

Max Activating Token Index: 408

Text #8

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.2375. Min Act: -0.0000

Data Index: 325120 (The Pile)

Max Activating Token Index: 586

Text #9

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.2653. Min Act: -0.0000

Data Index: 415910 (The Pile)

Max Activating Token Index: 477

Text #10

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.1324. Min Act: -0.0000

Data Index: 225952 (The Pile)

Max Activating Token Index: 224

Text #11

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.2864. Min Act: -0.0000

Data Index: 1968382 (The Pile)

Max Activating Token Index: 496

Text #12

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.2101. Min Act: -0.0000

Data Index: 1245649 (The Pile)

Max Activating Token Index: 984

Text #13

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.0928. Min Act: -0.0000

Data Index: 1283487 (The Pile)

Max Activating Token Index: 443

Text #14

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.1334. Min Act: -0.0000

Data Index: 993301 (The Pile)

Max Activating Token Index: 901

Text #15

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.1602. Min Act: -0.0000

Data Index: 1095900 (The Pile)

Max Activating Token Index: 939

Text #16

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.1550. Min Act: -0.0000

Data Index: 1596598 (The Pile)

Max Activating Token Index: 362

Text #17

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.1689. Min Act: -0.0000

Data Index: 1206500 (The Pile)

Max Activating Token Index: 379

Text #18

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.0720. Min Act: -0.0000

Data Index: 1387687 (The Pile)

Max Activating Token Index: 184

Text #19

Max Range: 1.3170. Min Range: -1.3170

Max Act: 1.1358. Min Act: -0.0000

Data Index: 1618648 (The Pile)

Max Activating Token Index: 183