Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1655 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.7570. Min Act: -0.0000

Data Index: 893165 (The Pile)

Max Activating Token Index: 386

Text #1

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.7221. Min Act: -0.0000

Data Index: 1876744 (The Pile)

Max Activating Token Index: 263

Text #2

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.6697. Min Act: -0.0000

Data Index: 118948 (The Pile)

Max Activating Token Index: 372

Text #3

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.6473. Min Act: -0.0000

Data Index: 582397 (The Pile)

Max Activating Token Index: 281

Text #4

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.6397. Min Act: -0.0000

Data Index: 443406 (The Pile)

Max Activating Token Index: 529

Text #5

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.6456. Min Act: -0.0000

Data Index: 415643 (The Pile)

Max Activating Token Index: 64

Text #6

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.5896. Min Act: -0.0000

Data Index: 650093 (The Pile)

Max Activating Token Index: 713

Text #7

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.5729. Min Act: -0.0000

Data Index: 1888654 (The Pile)

Max Activating Token Index: 968

Text #8

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.6257. Min Act: -0.0000

Data Index: 593779 (The Pile)

Max Activating Token Index: 170

Text #9

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.6214. Min Act: -0.0000

Data Index: 1617370 (The Pile)

Max Activating Token Index: 498

Text #10

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.5817. Min Act: -0.0000

Data Index: 78950 (The Pile)

Max Activating Token Index: 32

Text #11

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.5813. Min Act: -0.0000

Data Index: 635047 (The Pile)

Max Activating Token Index: 604

Text #12

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.6132. Min Act: -0.0000

Data Index: 510985 (The Pile)

Max Activating Token Index: 121

Text #13

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.5727. Min Act: -0.0000

Data Index: 421920 (The Pile)

Max Activating Token Index: 231

Text #14

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.5452. Min Act: -0.0000

Data Index: 403596 (The Pile)

Max Activating Token Index: 601

Text #15

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.5617. Min Act: -0.0000

Data Index: 76542 (The Pile)

Max Activating Token Index: 984

Text #16

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.5789. Min Act: -0.0000

Data Index: 665902 (The Pile)

Max Activating Token Index: 833

Text #17

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.5552. Min Act: -0.0000

Data Index: 1017750 (The Pile)

Max Activating Token Index: 174

Text #18

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.5919. Min Act: -0.0000

Data Index: 459471 (The Pile)

Max Activating Token Index: 230

Text #19

Max Range: 0.7570. Min Range: -0.7570

Max Act: 0.5200. Min Act: -0.0000

Data Index: 297530 (The Pile)

Max Activating Token Index: 386