Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2049 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.3401. Min Act: -0.0000

Data Index: 326294 (The Pile)

Max Activating Token Index: 293

Text #1

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2868. Min Act: -0.0000

Data Index: 1678504 (The Pile)

Max Activating Token Index: 71

Text #2

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2851. Min Act: -0.0000

Data Index: 1717586 (The Pile)

Max Activating Token Index: 539

Text #3

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.3133. Min Act: -0.0000

Data Index: 531083 (The Pile)

Max Activating Token Index: 369

Text #4

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.3011. Min Act: -0.0000

Data Index: 1901165 (The Pile)

Max Activating Token Index: 403

Text #5

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2764. Min Act: -0.0000

Data Index: 1887130 (The Pile)

Max Activating Token Index: 567

Text #6

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2766. Min Act: -0.0000

Data Index: 513421 (The Pile)

Max Activating Token Index: 508

Text #7

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2658. Min Act: -0.0000

Data Index: 57588 (The Pile)

Max Activating Token Index: 906

Text #8

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2905. Min Act: -0.0000

Data Index: 1126213 (The Pile)

Max Activating Token Index: 494

Text #9

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2619. Min Act: -0.0000

Data Index: 883924 (The Pile)

Max Activating Token Index: 931

Text #10

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2711. Min Act: -0.0000

Data Index: 1821996 (The Pile)

Max Activating Token Index: 400

Text #11

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2769. Min Act: -0.0000

Data Index: 1104851 (The Pile)

Max Activating Token Index: 493

Text #12

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2464. Min Act: -0.0000

Data Index: 581100 (The Pile)

Max Activating Token Index: 782

Text #13

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2680. Min Act: -0.0000

Data Index: 718519 (The Pile)

Max Activating Token Index: 338

Text #14

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2589. Min Act: -0.0000

Data Index: 1572180 (The Pile)

Max Activating Token Index: 73

Text #15

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2763. Min Act: -0.0000

Data Index: 739952 (The Pile)

Max Activating Token Index: 805

Text #16

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2381. Min Act: -0.0000

Data Index: 1940942 (The Pile)

Max Activating Token Index: 183

Text #17

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2626. Min Act: -0.0000

Data Index: 1743202 (The Pile)

Max Activating Token Index: 156

Text #18

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2624. Min Act: -0.0000

Data Index: 278102 (The Pile)

Max Activating Token Index: 419

Text #19

Max Range: 0.3401. Min Range: -0.3401

Max Act: 0.2571. Min Act: -0.0000

Data Index: 462973 (The Pile)

Max Activating Token Index: 472