Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1809 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.3754. Min Act: -0.0000

Data Index: 1462885 (The Pile)

Max Activating Token Index: 73

Text #1

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.3675. Min Act: -0.0000

Data Index: 1896440 (The Pile)

Max Activating Token Index: 115

Text #2

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.3043. Min Act: -0.0000

Data Index: 1538691 (The Pile)

Max Activating Token Index: 544

Text #3

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.3367. Min Act: -0.0000

Data Index: 666798 (The Pile)

Max Activating Token Index: 916

Text #4

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.3231. Min Act: -0.0000

Data Index: 1297766 (The Pile)

Max Activating Token Index: 958

Text #5

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.2780. Min Act: -0.0000

Data Index: 498814 (The Pile)

Max Activating Token Index: 304

Text #6

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.2666. Min Act: -0.0000

Data Index: 1635284 (The Pile)

Max Activating Token Index: 546

Text #7

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.2654. Min Act: -0.0000

Data Index: 1103850 (The Pile)

Max Activating Token Index: 136

Text #8

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.2476. Min Act: -0.0000

Data Index: 678617 (The Pile)

Max Activating Token Index: 341

Text #9

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.2561. Min Act: -0.0000

Data Index: 1930576 (The Pile)

Max Activating Token Index: 354

Text #10

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.2596. Min Act: -0.0000

Data Index: 1073245 (The Pile)

Max Activating Token Index: 621

Text #11

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.2429. Min Act: -0.0000

Data Index: 1572158 (The Pile)

Max Activating Token Index: 377

Text #12

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.2495. Min Act: -0.0000

Data Index: 203962 (The Pile)

Max Activating Token Index: 251

Text #13

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.2546. Min Act: -0.0000

Data Index: 1186224 (The Pile)

Max Activating Token Index: 350

Text #14

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.2611. Min Act: -0.0000

Data Index: 1859720 (The Pile)

Max Activating Token Index: 276

Text #15

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.2441. Min Act: -0.0000

Data Index: 467532 (The Pile)

Max Activating Token Index: 23

Text #16

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.2629. Min Act: -0.0000

Data Index: 1240633 (The Pile)

Max Activating Token Index: 373

Text #17

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.2439. Min Act: -0.0000

Data Index: 128438 (The Pile)

Max Activating Token Index: 841

Text #18

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.2021. Min Act: -0.0000

Data Index: 1567736 (The Pile)

Max Activating Token Index: 776

Text #19

Max Range: 0.3754. Min Range: -0.3754

Max Act: 0.2372. Min Act: -0.0000

Data Index: 1210679 (The Pile)

Max Activating Token Index: 425